Imagine, as I sometime ask, that you are doing an experiment which is very sensitive to external magnetic fields. (Like, oh, I don’t know, an atomic clock). And you find evidence of some stray field gremlin that has taken up residence. Since every previous time this has happened it has been the result of a thermoelectric current, you might be lulled into thinking that it’s the same thing, and fixing it will be a piece of cake. A colleague might even announce something to that effect: “It’s a current. It’s always a current. If there’s one thing you can depend on, it’s that stray fields are always currents.”
Welcome to the phenomenon of the sportscaster’s curse.
The sportscaster’s curse, in case you’re not familiar with it, is a phenomenon seen during sportscasts, in which the announcer will basically guarantee an outcome, which then dooms the effort to failure. The athlete is tagged as “Mister Automatic” in some way, with a mention of how he hasn’t missed a free throw/short putt/chip-shot field goal in X attempts, at which point the attempt clangs off the upright or rim, or lips out of the cup. (I’m sure a fair bit of confirmation bias is present here, since the curse doesn’t strike every time, but I cringe nonetheless if it’s a player on my team being lauded for his reliability)
So this is what happened. After we convinced ourselves that it was a simple problem and a quick fix, as happened earlier, we took all the steps to fix it. These steps include taking off the nested layers of magnetic shielding which make the fountain look like a Russian-doll hot-water heater. (or really just a water heater, because you don’t need to heat the water if it’s already hot)
(Such a device might look something like this)
Nada. No obvious connections. We let it cool down to room temperature to minimize gradients and reassembled, but there was no change in the signal. OK, disassemble again and start checking for some magnetic component. But we’re looking for a milligauss-ish field, which isn’t going to be seen amidst the half a gauss of the earth’s field, so the only real way to do this systematically is to change one thing and reassemble it so we can look at the signal in a shielded environment.
We did that a lot over the past few days.
We finally decided that looking with a magnet might be a good idea — a strong one might stick to the offending component. There shouldn’t be any downside — the nonmagnetic materials aren’t going to become magnetized, and if there is a magnetic part, it will only change the scale of the already-existing problem. The latter is exactly what happened. The magnet didn’t stick to anything, but all of the sudden (after yet another reassembling and degaussing of the shields) the problem was much bigger — we had induced more magnetization, and that made it easier to find the offending component. Kinda like finding a needle in a haystack by being able to make the needle a lot bigger.
It was the salmon mousse a washer on a bolt in the vacuum system. Somehow a shiny stainless steel washer had successfully been hiding among the copper ones, and nobody noticed; it either had acquired a similar-looking tarnish, or because of the shininess it looked coppery when it was in the bin. In any event, transplant surgery was indicated and carried out successfully without a vacuum breach (which is good because losing vacuum would have sucked in all the wrong ways)
i know about how breaking vacuum sucks. our chambers have been having problems recently and have necessitated multiple vents and bakes after repairs. now the most recent issue is one of our chambers seems to have a leak. at first it was only nitrogen, which could be from our LN2 cooling panels, but yesterday oxygen appeared too–which means air. now we gotta look for a smaaaaaallllll leak. somewhere. argh.
good catch on the bolt!
Several days work because a steel washer fell into the copper washer box – makes me glad I am an enthusiastic amateur observer rather than a researcher.
Awesome story! It’s always the “little things” that get you. Nice detective work.