There used to be a huge temptation to go to site and get hands on with things. A friend of mine seemed to particularly like to do this rather than use the management capabilities of the devices, at least that’s how I remember it.
In this particular instance he decided to go to site for whatever reason, possibly to coordinate better with the highly non-technical fault reporter whilst looking at our solution from end to end at the same time. Often a very valid way to fault find, albeit time consuming, costly and a bit annoying.
On return he looks a bit grumpy and tells me how on inserting a console cable into the onsite device, it rebooted, taking the whole office down for a few minutes. Then it was fine, right up until he removed the console cable and it rebooted again.
This sort of thing you just can’t make up. It took a long time in the office with one of the devices to work out it was a specific combination of USB to serial adapter, laptop, cable and software version to guarantee this would happen.
Learning point for everyone:
There is a common theme in some of this commentary around obscure faults. It’s that heady combination of physical presence, unexpected behaviour and an outage.With the advent of cloud computing and a move to general acceptance that remote consoles are normal, combined with less and less equipment on site incidents such as the above should, in theory, become less and less likely.