Hi!

I am trying to learn  DSP programming, so I decided to implement some demodulations on Rtlsdr dongle. I managed to do AM and FM demodulation, and everything works fine, however there is a lot of noise in the signal (for both demodulations) comparing to SDR# demodulations (especially for AM when I am using direct sampling).

My AM demodulator works like this:

Read_rtlsdr()-->lowpass_filter_4000Hz-->resample-->demodulate (sqrt(q*q+i*i))--> removeDC-->scale-->audio_out
What  blocks should I add?

For now I am planning to add squelch, does anybody know a good reference/example to see how this  should be done?

It looks like a reasonable block diagram.  Doing some low-pass filtering after the demodulate step might help a little, but I'd be surprised if it made your problems go away.  Synchronous demodulation might be slightly better, but again I wouldn't expect lots of noise removal over what you're doing, just some, and better performance when you're in the fringe reception area for a given station.

Can you post the frequency response of your 4kHz lowpass, and your sampling rate after resampling?

Does your noise sound like random noise (i.e., rain on a tin roof), or does it sound like chirping or thumping or whistling or some such?

I can try to plot freq. response of a filter tomorrow. It is a FIR filter calculated using sync function (impulse length is 30), and I also take care of the parts of the signal that are not usable after the convolution. I am primarily sampling at 1058400 Hz and then decimate by factor 23. Audio sampling fr. is 44100 Hz.

Noise is more random I would say, and there is constant high pitch component with it. The noise its quite loud and  then in the background there is quite clear but silent radio signal. 

What can I conclude from different types of noise?

I also tried filtering the audio signal afterwards but as you predicted there is no not much change.

So, you're making a FIR filter 30 taps long?  That's not nearly long enough for decimation by a factor of 23.  I'm guessing something that's 200 taps long or more if designed using the Remez exchange algorithm, and longer yet if you use a sinc.

I think that what you're hearing is nearly the entire AM band aliased into your one signal.  The random noise is either atmospheric or a lot of radio stations piled on top of each other, the high whistling is the carrier of some radio station, aliased.

Read this: http://www.wescottdesign.com/articles/Sampling/sam....  Then look at your lowpass filter's response and see if you can figure out what's happening.

Hi. Regarding AM demodulation, perhaps the following blog would be of some interest to you:

https://www.dsprelated.com/showarticle/938.php

For the great answers you already got I just wanted to add a another small issue.

I also worked a little with the SDR dongles and one problem I tried avoiding is to use the direct conversion at the exact frequency due to DC leakage. 

To remove the DC you can either add a high pass filter or just sample the signal around some higher frequency IF, like 100KHz and down convert it from there using the DSP. 

The high pass filter will have to remove only the very low frequencies and so may be very long and will take a long time to converge.