I know that for stationary data, the ACF function should die down fast. But I don't know how fast. How the significance level (95%) is calculated to compare if ACF function is decaying soon enough to declare the time series data as stationary. 

In short, how can I check if my noise data is stationary using ACF? 

Thanks in advance

Let the data length be 2N samples.Then centered at sample N/2 find auto correlation for

0 to N-1 sample.Similarly centered at N+N/2 find Auto correlation for N/2 samples either

side.If both are same it is Wide Sense Stationary.Reason is for a shift in time auto correlation should not vary for W.S.S.

Unless the process is something like electrostatic noise, where there's a burst of noise any time there's a spark someplace (i.e., a lightning strike near or far).  So if you get a hit in one interval but not another then you fail the test, and yet the overall noise statistics are independent of time.

You could resolve it by taking the autocorrelation over lots and lots of intervals.

Theoretically, stationarity means that  the joint distribution of any collection of the time series variates never depends on time. So, the mean, variance and any moment of any variate is the same whichever variate you choose.

However, this too strict to use in practice. So people have used  the second-order stationarity instead of the strict stationarity by checking that

1) mean function of the time series = constant over time

2) ACF of the time series --> a function of only time difference

How to measure the difference of ACF? In my case, I have used KL distance. From my experience, the KL distance works!.

KL distance?

It looks like Jaewook means he adapted this to use with the ACF:

http://users.monash.edu/~lloyd/tildeMML/KL

https://stats.stackexchange.com/questions/14673/me...

Although I can't find a discussion of this combination for noise characterization.  Maybe worth trying.

The only thing that stationarity means is that absolute time isn't a factor in the statistics of the noise.  You could have an autocorrelation function that takes longer than our lifetimes to die off, and still have a process that, in a strict sense, is stationary.

Having said that, if you had such a process and you were using its statistics over some span shorter than your lifetime, you could approach it as a non-stationary process by starting from its known behavior over the span of seconds, hours, days or weeks.