Hi everyone ,

I'm looking for ideas in order to implement an algorithm that would be able to compare these 2 plots, these are 3D data. I work with Matlab.

Of course I already did simple substraction and cross-correlation. But I would like to exploit something more complex or imaginative. 

If you got anything do not hesitate ! As mentioned,I work with Matlab but if something exists on another software I will be still interested to look at it :)

edit : "The output should be something that give how big is the difference and also where it is. But also, there is a shift (No obvious on the 3D plot I agree), how could it be possible to quantity this shift."  See more details below.

Thanks !!!

Others have given useful comments.  Wanting something "more complex" seems to go against Einstein's "everything should be made as simple as possible, but no simpler" and  Voltaire's "better is the enemy of good enough".

But since we don't know the objective, I'll try to state one:

"Compare the two data sets in order to determine their similarities or common components".  This may include aligning them in space; i.e. where is the peak of the correlation?

So, subtraction won't do it.  It determines their differences.

But, if we were to say:

"Compare the two data sets in order to determine their differences or unique components" then subtraction might be more interesting.

And, of course, if we were to say:

"Compare the two data sets in order to determine common components and unique components" then a variety of approaches seems indicated.

Here are some ideas:

- compute the 2-D Fourier Transform and multiply (or some other operation).  This takes but a little padding to avoid spatial aliasing.  Granted, it's similar to correlation in the spatial domain but it may lend insight or measures not heretofore considered by looking at the spatial frequencies.

- split the data sets into signal + noise using a common (i.e. the same) algorithm.  Compare the signal parts only.

It would probably help a lot if we knew how the data sets were generated and more about your objective(s).

Well, I would like to give you more specificities but it is confidential so I really try my best to overcome this issue without finishing in jail haha. Sorry...

"This may include aligning them in space" -> Correlation at 0 lags right?

I will try to compute the 2-D Fourier Transform asap. (Tomorrow probably) I keep you in touch. However, I did not understand the precise meaning of :"similar to correlation in the spatial domain but it may lend insight or measures not heretofore considered by looking at the spatial frequencies.", but I guess I should better try to execute the code before. 

"split the data sets into signal + noise" There should be no noise into the data. It has been recorded with a statistical removing noise procedure. There are from a recording.

objective(s) : "Compare the two data sets in order to determine common components and unique components" This is a really accurate formulation of what I want. 

Nice French reference btw, 'Le mieux est l'ennemi du bien’ in my native langage! Thank you a lot for your very useful feedback. 

as far as I know Fourier transform followed by multiplication (plus reversal) is equivalent to time domain correlation if done with correct resolution.

Kaz

Yes.  But if you don't do the reversal, there is an opportunity to see the result.  That can give insight.

Hello,

Only if you tell what complex or imaginative things you have in mind, may be I can help. There is a lot you can do with MATLAB.

Cheers!

What sort of features or properties are you trying to reveal?  "Compare" is pretty broad.

What the other two guys said: without understanding what you're looking for and why no one is going to be able to come up with something that'll give meaningful results.  "Complex and "imaginative" are what you do when the boring old stuff fails, but if there's know understanding of what you're looking for, they're just a more expensive failure with bigger words.

I think the request of the op is clear. I have no idea apart from correlation but you can also divide into tiles and correlate tile by tile as well then sort out some weighting factor.

Kaz

Hey Kaz. You mean like a kind of 2D image cross correlation pixels by pixels with some weighting ? But instead of defining pixels you just proceed with independent, rectangular regions. 

indeed. You may also correlate each and every tile with each of all other tiles just to get more info.

Kaz

Well, it's not at all clear to me.  Tell me what OP wants to know from the comparison, then.  

He doesn't even say if he's interested in "match" vs. "no match" or "pretty" vs. "ugly", he doesn't mention the nature of the signal or the nature of the differences that might arise between the graphs, even to the point of knowing if those differences can be aptly described as "noise", or just about anything.

I understand that it might sound confusing. Please have a look at Fred Marshall response he understood well what I want.

"match" vs. "no match" -> Not my concern, I may compute it later but I guess it's rather simple to perform a Y/N matching.

"pretty" vs. "ugly" -> There should be no noise into this kind of mountains of data. The data-acquisition disposes of a statistical noise removing procedure. The nature of such a difference is unknown, that why I look at the similarities and differences of it.  

Thank for your response all of you. (@charansai  @Tim Wescott @Slartibartfast)

The point is to quantify how much these 2 results are different, how could we quantify it and extract in a smart way the features. 

The output should be something that give how big is the difference and also where it is. But also, there is a shift (No obvious on the 3D plot I agree), how could it be possible to quantity this shift. 

Unfortunately I'm not sure if I can reveal the X-axis (confidential, that why I can not share everything even if I would like to...) but the Y-axis is a frequency axis so no need to switch to Fourier domain. 

I thought about doing a simple absolute comparison (See below - This plot does not correspond my first message plot examples - Careful) and then operate and adaptative segmentation based on the change of amplitude.

Perhaps if the objectives of this "comparison" could be stated clearly, that would lead directly to a course of action. Above, you stated "shifted" but even that is too general (e.g. shifted in amplitude, rotation, phase, whatever). Are all axes normalized? Is there noise? Is it Gaussian or not (if not, then correlation may be unreliable).

Or maybe the answer is the result ascribed in the "Hitchhiker's Guide to the Galaxy" of "42" (of maybe its unwritten sequel "..to the Universe").

Like too many problems, 99% of a solution is a concise and unambiguous statement of the problem. Keep defining the problem and its solution should reveal itself.