-
#3880 by Rodal on 15 Jul, 2016 11:54
-
ALL moving averages done in real time are trailing, because the numerical smoothing can only process past data up to the present (for a real-time scheme, future data is in the future and hence unknown by a smoothing scheme done in real time. The only way to have a "central" moving average is to know future data, and that can only be done once the test is finished, at which point what was once future data has become past data.
Actually the fact that they are "trailing" in nature is a problem with all smoothing schemes done in real time, not just moving averages. It doesn't matter whether it is a moving average based on the last N data, or an exponential moving average or any other such scheme. The trailing problem has to do with the fact that the smoothing scheme can only process past data.
I agree with RERT that if the moving average is speculated to be the source of an artifact, it should be turned OFF, to see what the data looks like with the moving average completely turned off. If the data looks too jagged with the moving average turned off, then moving averages with shorter length of N data should be explored until a compromised is reached between the bad features of moving averages (its trailing nature and artifacts like flat areas) and the need for smoothing.
Of course, certain kinds of smoothing schemes are more prone to flat areas, for example a moving median is more prone to flat areas than a moving mean, because the median as statistical measure of central tendency is unaffected by extreme data while the mean even takes into account outliers. -
#3881 by rfmwguy on 15 Jul, 2016 12:06
-
At the risk of sounding obvious, perhpa the simpler solution is a horizontal extension of the moment arm for the LDS target. A hollow aluminum tube double the length of the existing arm does 2 things: increases displacement for a given force and further isolates the LDS head. I''ve avoided too many suggestions as it can be frustrating to sort through all of them, but the simpler the better has been my game plan. Elegant? Maybe not. Effective? Probably.
-
#3882 by Rodal on 15 Jul, 2016 12:14
-
At the risk of sounding obvious, perhpa the simpler solution is a horizontal extension of the moment arm for the LDS target. A hollow aluminum tube double the length of the existing arm does 2 things: increases displacement for a given force and further isolates the LDS head. I''ve avoided too many suggestions as it can be frustrating to sort through all of them, but the simpler the better has been my game plan. Elegant? Maybe not. Effective? Probably.
It should be taken into account that a horizontal extension will increase the moment of inertia:
dramatically, as the moment of inertia goes like the square of the distance, while the displacement only increases linearly with the length.
Drastically increasing the moment of inertia, drastically increases the inertial forces for the same rotational acceleration. It drastically decreases the natural frequency and it therefore increases the period of the pendulum (if the torsional spring constant is kept the same). It has a similar effect on a torsional pendulum (except that it goes like the square of the length), as increasing the mass in mass-spring resonant system.
Hence, it should be taken into account that a horizontal extension of the beam will change the present dynamics of the torsional pendulum (increasing the period). In other words, when the torsional pendulum gets perturbed from its position of equilibrium, it will respond more slowly, the period of natural oscillations will be longer. (Since the period goes like the square root of the moment of inertia, and the moment of inertia goes like the square of the length, the period is a linear function of the length). -
#3883 by rfmwguy on 15 Jul, 2016 12:23
-
Understand. A longer period is good and bad. Long oscillations are easier to distinguish from other forces yet impedes short duration events. In the case of the emdrive, instantaneous, short duration displacement is less interesting than a sustained force, so a trade off occurs. Sacrifice moment of inertia for higher displacement and isolation is my recommendation.At the risk of sounding obvious, perhpa the simpler solution is a horizontal extension of the moment arm for the LDS target. A hollow aluminum tube double the length of the existing arm does 2 things: increases displacement for a given force and further isolates the LDS head. I''ve avoided too many suggestions as it can be frustrating to sort through all of them, but the simpler the better has been my game plan. Elegant? Maybe not. Effective? Probably.
It should be taken into account that a horizontal extension will increase the moment of inertia:
dramatically, as the moment of inertia goes like the square of the distance, while the displacement only increases linearly with the length.
Drastically increasing the moment of inertia, drastically increases the inertial forces, hence drastically decreasing the period of the pendulum (if the torsional spring constant is kept the same). It has a similar effect on a torsional pendulum (except that it goes like the square of the length), as increasing the mass in mass-spring resonant system. -
#3884 by Monomorphic on 15 Jul, 2016 13:18
-
Pretty sure I have the flat artifacts problem figured out. I was using the span adjustment incorrectly.
I had the span turned way down. This made the signal span too narrow for the resolution of the ADC.
I am also purchasing an aluminum box to house the omron amp and ADC. Will also try and shield the LDS itself.
I should have MUCH better data soon. -
#3885 by frobnicat on 15 Jul, 2016 13:30
-
ALL moving averages done in real time are trailing, because the numerical smoothing can only process past data up to the present (for a real-time scheme, future data is in the future and hence unknown by a smoothing scheme done in real time. The only way to have a "central" moving average is to know future data, and that can only be done once the test is finished, at which point what was once future data has become past data.
Since we (at least I) suspect the averaging is a software post process, it can very well be done as central "non causal, none time shifting". Maybe the numerical "noise cancelling" is done by the hardware itself, maybe there is a buffer of the length of N samples, who knows, but the potential time shifting introduced seems to me a minor problem compared to the severe range mismatch analog output of LDS typical 160mV (given the amplitude of movements in experiment) vs 20V full scale input of 10 bits ADC.QuoteActually the fact that they are "trailing" in nature is a problem with all smoothing schemes done in real time, not just moving averages. It doesn't matter whether it is a moving average based on the last N data, or an exponential moving average or any other such scheme. The trailing problem has to do with the fact that the smoothing scheme can only process past data.
I agree with RERT that if the moving average is speculated to be the source of an artifact, it should be turned OFF, to see what the data looks like with the moving average completely turned off. If the data looks too jagged with the moving average turned off, then moving averages with shorter length of N data should be explored until a compromised is reached between the bad features of moving averages (its trailing nature and artifacts like flat areas) and the need for smoothing.
Flat areas are not an artefact of an hypothetical "noise cancelling" numerical post process of a moving average kind (whatever kind). It is an artefact of the ADC input range mismatch with the analog signal being sampled. If anything the post-process erodes the flat areas and blurs the transitions, it only serves to hide the fact, but can't re-invent lacking precision. Although in those matters there is possible time precision vs value precision tradeoff (by low pass filtering) a carefully calibrated analog noise of precise amplitude on top of the analog signal could in principle be filtered later on (with a loss on time resolution) and additional bits of precision "added" on values (a kind of stochastic interpolation) without introducing severe distorsion at native discrete steps, but that is quite a stretch.QuoteOf course, certain kinds of smoothing schemes are more prone to flat areas, for example a moving median is more prone to flat areas than a moving mean, because the median as statistical measure of central tendency is unaffected by extreme data while the mean even takes into account outliers.
Yes but again flatness comes first from the range mismatch input of ADC, later smoothing can only more or less aptly obfuscate that. A proper analog signal pre-amplification (before feeding the ADC) raising the 160mV typical amplitude of output of the LDS to some 0...5V would give 8 bits of precision instead of approx. 3, removing all the flat transitions (at least as seen on a chart with 256 vertical pixels). Ideally one would want to tune analog amplification of signal to maximize (within some margin) the use of the full scale input range -10V ... +10V of ADC. A +-15V or +-12V fed op-amp can easily provide both scale and offset for such purpose with probably minimal noise and distortion, much much better than running directly the signal at less than 1% of the nominal digitization value range (and expecting miracles from numerical post treatment) !
-
#3886 by rfmwguy on 15 Jul, 2016 13:38
-
Pretty sure I have the flat artifacts problem figured out. I was using the span adjustment incorrectly.
Ahhh, the good old days of setting up the LDS last August...good job! I had the span turned way down. This made the signal span too narrow for the resolution of the ADC.
I am also purchasing an aluminum box to house the omron amp and ADC. Will also try and shield the LDS itself.
I should have MUCH better data soon. -
#3887 by Rodal on 15 Jul, 2016 13:52
-
When is the smoothing done? Is it done in real-time for display purposes, or is it done only after ALL the data has been gathered, after the whole experiment has been finalized?ALL moving averages done in real time are trailing, because the numerical smoothing can only process past data up to the present (for a real-time scheme, future data is in the future and hence unknown by a smoothing scheme done in real time. The only way to have a "central" moving average is to know future data, and that can only be done once the test is finished, at which point what was once future data has become past data.
Since we (at least I) suspect the averaging is a software post process, it can very well be done as central "non causal, none time shifting"....
One cannot, in general, smooth data in real-time, without having a trailing problem, since as I said, in real-time the software only knows the past (*). All smoothing algorithms in real-time dealing with processes where the future cannot be forecasted have this trailing problem. It is a well-known problem in control, involving random data and nonlinearities, as one cannot forecast the unknown future. Hence, no, for a process where the future data is unknown you cannot get away from the time-shifting problem. You could only do that in real-time for a process where the future can be perfectly forecasted based on past data. Or, when the experiment is completely done, and dealing with all the data at once, such that there is no unknown future data.
In these experiments, where people are trying to measure a process which is unexplainable according to classical physics, and it is debated whether the anomalous force measurement is a result of experimental artifacts or an unknown physical process, there is no algorithm that can forecast the future in real time (*), and hence there is no algorithm that can smooth the data based on past data that will not display a trailing, time-shifting problem due to smoothing of past data.
More basically, for example if it is up to Monomorphic to turn the power on and off, using his freedom of choice, or to turn the power to less than 100% power, or to jump in the room, or somebody walks into the room, or if there is a draft of air due to an unforecastable event, no algorithm will know ahead of time what Monomorphic may decide to do with his free-will, or an unforecastable event may occur. Algorithms can only forecast data that obeys a deterministic process, for example the dynamics of the pendulum as long as it is deterministic.
However, you yourself described the response behavior of Monomorphic's data as "chaotic", if there is chaos involved, then there is no algorithm that will be able to forecast the chaotic behavior either, because of exquisite sensitivity to initial conditions. hence you are back to the problem of smoothing in real-time and hence condemned to having a trailing moving measure of central tendency, if one wants to smooth the data in real-time, based on past data.
---------------------------------------
(*) of course, once the experiment is done, then all data is in the past, and it is trivial at that point to smooth the data such that there is no trailing problem, as there is no unknown future data at that point in time, as all data is known at that point in time. -
#3888 by rfmwguy on 15 Jul, 2016 14:53
-
After we formally signed a permanent EmDrive truce on the ISS last week
, IslandPlaya on reddit's EmDrive sub has been working diligently to put together a helpful Cavendish software package you may find interesting. As with DIYers, its a work in progress but I'm excited to have this being worked on.
Modeling test stand performance from a skeptic's point of very I think is extremely valuable. Here is his first cut based on my rough test stand dimensions.
IslandPlaya's First Cut:
Defective Magnetron Test:
Last Data Run with Working Magnetron - Note Thermal Superimposition
I see very close correlation to the ADC output charts!
Original post: https://www.reddit.com/r/EmDrive/comments/4sz9a2/emdrive_torsionbalance_experiment_simulator/ -
#3889 by frobnicat on 15 Jul, 2016 15:09
-
Yes dr Rodal, I agree in whole with what you say about causal filtering possible effect on undesirable timing uncertainty : this matter needs to be addressed either by characterizing such time shift or (preferably) turning off "black box" software or firmware numerical post treatment to work on raw data (at hardware temporal sampling resolution). Also being the same device digitizing both signals of LDS and indications of power on/off (is that so ?) they likely introduce the same (possible) time shift on both channels, so this is neutral for interpretation of plots.
My argument was that the huge range mismatch was causing severe quantization distortion and lack of real precision in values and that needed to be addressed first because introducing false shapes altogether, at a level much worse than just possible time lag of 16 successive samples (with correct shapes). Now that monomorphic has found the correct use of a span tuning knob hopefully this specific matter is improving and we may proceed to further question precision of timing. Also that draws our attention to range matching and real numerical precision : 10 bits ADC does not automatically means 1024 digitized levels of precision across the plots when working on a sub-optimal analog range.
Monomorphic, is the span adjustment on the LDS or on the ADC ? Have you a way to tell that the previously typical 160mV analog range (within bounds of typical experiment displacements) is now closer to -10V +10V full scale of ADC ? Is there any on-by-default "smoothing" or "low pass" or "noise cancelling" setting in the acquisition software ? -
#3890 by Rodal on 15 Jul, 2016 15:14
-
...
Last Data Run with Working Magnetron - Note Thermal Superimposition
I see very close correlation to the ADC output charts!
Original post: https://www.reddit.com/r/EmDrive/comments/4sz9a2/emdrive_torsionbalance_experiment_simulator/
Why does the torque increase with time (the red bars), when the input power is not being increased?
Notice: the image was labeled "with Working Magnetron"
-
#3891 by rfmwguy on 15 Jul, 2016 15:57
-
Input power is neither instantaneous/constant nor is the frequency stable. Center moves toward resonance as it heats up:...
Last Data Run with Working Magnetron - Note Thermal Superimposition
I see very close correlation to the ADC output charts!
Original post: https://www.reddit.com/r/EmDrive/comments/4sz9a2/emdrive_torsionbalance_experiment_simulator/
Why does the torque increase with time (the red bars), when the input power is not being increased?
Notice: the image was labeled "with Working Magnetron"
-
#3892 by zen-in on 15 Jul, 2016 16:00
-
Zen_in : my explanation (moving average) why there are intermediate levels between flat areas is speculation because I don't know what does the software after digital conversion, but even if we had not seen any plot or take any measurements we could know beforehand that :
The digitized signal is spanning 160 mV of range, the hardware ADC converter is documented as being 10 bits on a range of 20V, that makes discrete steps every 20V/(2^10)≈20mV,( that's your twice " approximately .01" separating each flattish levels), 160mV/20mV = 8 discrete levels to play with, 3 real hardware bits of precision. This is a rather direct and straightforward prediction of known parameters, even if we noticed the problem after looking at recorded value.
The LDS signal needs to be amplified and maybe a different A/D used. With no schematic or any technical description of the experimental setup I can only guess at what is being done wrong. For the LDS you don't want a bipolar A/D. Preliminary tests should establish if the signal going into the A/D is high enough so that digitization noise is not an issue but not so high that it exceeds the A/D range. It also might be necessary to have a well designed buffer before the A/D. Sensors often cannot supply the current spikes that an A/D's sample and hold pull. Without a buffer the signal gets distorted. -
#3893 by rfmwguy on 15 Jul, 2016 16:12
-
The LDS system does have a buffer amp the W40RA. The Omron Z4M is the LDS head.Zen_in : my explanation (moving average) why there are intermediate levels between flat areas is speculation because I don't know what does the software after digital conversion, but even if we had not seen any plot or take any measurements we could know beforehand that :
The digitized signal is spanning 160 mV of range, the hardware ADC converter is documented as being 10 bits on a range of 20V, that makes discrete steps every 20V/(2^10)≈20mV,( that's your twice " approximately .01" separating each flattish levels), 160mV/20mV = 8 discrete levels to play with, 3 real hardware bits of precision. This is a rather direct and straightforward prediction of known parameters, even if we noticed the problem after looking at recorded value.
The LDS signal needs to be amplified and maybe a different A/D used. With no schematic or any technical description of the experimental setup I can only guess at what is being done wrong. For the LDS you don't want a bipolar A/D. Preliminary tests should establish if the signal going into the A/D is high enough so that digitization noise is not an issue but not so high that it exceeds the A/D range. It also might be necessary to have a well designed buffer before the A/D. Sensors often cannot supply the current spikes that an A/D's sample and hold pull. Without a buffer the signal gets distorted. -
#3894 by Tellmeagain on 15 Jul, 2016 18:12
-
Last Data Run with Working Magnetron - Note Thermal Superimposition
I see very close correlation to the ADC output charts!
Original post: https://www.reddit.com/r/EmDrive/comments/4sz9a2/emdrive_torsionbalance_experiment_simulator/
I suggest a control experiment to this one. Keep everything intact, but disconnect the high voltage lead on the microwave oven side (keep the filament leads intact). Now run the experiment with the same time course as the pictured original experiment above. Thanks!
-
#3895 by Monomorphic on 15 Jul, 2016 20:56
-
The same flattened areas are in Dave's data. You just need to zoom in to see it as with mine. It shows up at the scale ~0.1V in all the data i've seen. Though it looks like using the 3um (60ms) mode may yield the best data at the level.
-
#3896 by Monomorphic on 15 Jul, 2016 21:04
-
Sensor electronics shielding completed. I also moved my PC to the other side of the bench, it was practically beneath the emdrive before. Now it's about 9 feet away.
-
#3897 by Rodal on 15 Jul, 2016 21:11
-
The same flattened areas are in Dave's data. You just need to zoom in to see it as with mine. It shows up at the scale 0.1V in all the data i've seen. Though it looks like using the 3um (60ms) mode may yield the best data at the level.
One thing I don't think was answered was whether these data is the result of output with a device that is smoothing the data real-time.
And if the answer is yes, whether the real-time smoothing of data can be turned off.
-
#3898 by Monomorphic on 15 Jul, 2016 21:25
-
The same flattened areas are in Dave's data. You just need to zoom in to see it as with mine. It shows up at the scale 0.1V in all the data i've seen. Though it looks like using the 3um (60ms) mode may yield the best data at the level.
One thing I don't think was answered was whether these data is the result of output with a device that is smoothing the data real-time.
And if the answer is yes, whether the real-time smoothing of data can be turned off.
I think there is some real-time smoothing/rounding going on, but I'm not sure if it's the LDS or the ADC at this point. I've spent a large portion of the day trying to completely eliminate the flattened areas. The ADC is the cheapest component by far at $29 vs $1,500 for the LDS.
Here is a picture of the controls on the amp. This is all there is control-wise.
-
#3899 by Monomorphic on 15 Jul, 2016 21:44
-
It's got to be the ADC. From the datasheet: "Provides 19.5 mV resolution" This is what makes it look 8-bit at 0.1V!
And for its $250 big brother: 10mV maximum resolution
http://www.dataq.com/resources/pdfs/datasheets/di-245ds.pdf
And its $500 bigger brother: 1.2mV at +-10V and 122µV at +-1V
http://www.dataq.com/resources/pdfs/datasheets/710ds.pdf
I had the span turned way down. This made the signal span too narrow for the resolution of the ADC. 
, IslandPlaya on reddit's EmDrive sub has been working diligently to put together a helpful Cavendish software package you may find interesting. As with DIYers, its a work in progress but I'm excited to have this being worked on.