Multiple variable/correlation analysis using Python

Question

I have a machine with up to 58 indipendent input variables and one response variable. If I create 2D-histogram plots of all combinations of 2 input variables and the response variable, I get plots like these (the more light the color, the stronger is the response):

Note that the x and y variables are both independent variables, only the response indicated as color is dependent on the other two.

I am looking for a metric to describe the pinkish "blob" in these plots which I want to analyze using python. I am tinking in the direction of correlations or linear fits with R squared as metric, but I am not sure if this is the right approach.

Trying to fit an ellipse that contains them would be one way. This is probably a better fit on cross validated where the statistics and model fitters hang out. Are the bright yellow single pixels the optimum parameter setting? — Martin Brown
– Martin Brown, Commented Nov 24, 2025 at 16:30
Yes, the bright pixels are the optimum settings. The data comes from a particle accelerator. XY are voltages applied to electrodes, the Z-dimension is beam current. — WolfiG
– WolfiG, Commented Nov 24, 2025 at 16:56
You might find that simplex optimisation of the various parameters will help. I have found it useful in ion optics focussing problems for mass spectrometers which is not so different to your use case. Are the optima all as sharp as the samples you have shown? One yellow single pixel and a bunch of noise in an oval? I'd like to see a zoom in 2830-2900, -113, -103 in steps of 1v if possible. It looks to me like there may actually be a very sharp focus and the mush around it is due to scattering in the beam line. I like to see sharp peaks slightly oversampled (also a colour map key). — Martin Brown
– Martin Brown, Commented Nov 26, 2025 at 9:55

J_H · Accepted Answer · 2025-11-24 20:44:38Z

3

That 1st beam looks horizontally oriented, while the 2nd one is vertically oriented.

loss function

looking for a metric to describe the ... "blob" in these plots

It seems you're not quite ready for that. I don't know what your business Use Case is, but I suspect that it relates to providing a feedback signal to a control loop. In which case you would want to write down a Loss function that encodes the final result that you really care about. Maybe you want to measure the oblong beam's orientation θ, in which case fitting ellipse axes or even a thresholded bounding box could suffice. Or maybe you care about beam focus, in which case fitting a circle's radius, or size of a thresholded square, would work. Or maybe symmetry is really what matters, independent of the cross-sectional area.

The point is, there is something in this physical situation that matters to you, and you should write it down. That will pretty much directly give us a Loss function, and from there we can worry about efficiently computing it, hill climbing, and implementing a useful model.

answered Nov 24, 2025 at 20:44

J_H

21.8k5 gold badges29 silver badges58 bronze badges

Sign up to request clarification or add additional context in comments.

WolfiG Nov 25, 2025 at 12:39

Hi J_H, as far as I can see, I do not want a loss function. I'd like to find out how strongly the response signal Z is correlating with the input signals X and Y

J_H Nov 25, 2025 at 17:10

Well, it's just part of the nomenclature we find in the modeling literature. The loss function drives the learner as it fits model parameters. Consider a y=mx+b linear model, with 2DoF. The noisy experimental observations won't fall exactly on a line. We could minimize sum of residuals. We could even choose to insist that all residuals are positive (line below the observations) due to some business reason. But often we find that choosing to minimize sum of squares of residuals works better for our business problem and is robust to outliers. You need a model (circle?) and a Loss fn.

Collectives™ on Stack Overflow

Multiple variable/correlation analysis using Python

1 Answer 1

loss function

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

loss function

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related