A machine by its sound
Can KELVIN hear that a machine is not running normally, when it has heard only the machine's normal running, and with no pre-training?
Benchmark

The question
Can KELVIN tell from its sound that a machine is not running normally, when it has heard only the machine's normal running, and with no pre-training?
What was done
KELVIN learned each machine from 100 normal recordings per section, with no pre-training, on a CPU and no graphics card, with other work running on the same computer. A first listen took a median 311 ms per recording, a finer one 1 268 ms, and a decision 293 to 523 ms.
The result
On the DCASE 2022 Challenge Task 2 evaluation set, five real machine types (fan, gearbox, bearing, slider, valve), KELVIN scores 56.50 by the challenge's own evaluator code. The official baselines published 52.94 (an autoencoder) and 54.02 (MobileNetV2), over seven machine types. The bar we registered before the run, above both baselines on at least 4 of 5 machines, was missed: 3 of 5.
Per machine, KELVIN's AUC is 51.41 on the fan, 64.61 on the gearbox and 64.00 on the bearing, above both official baselines; on the slider 61.35, where MobileNetV2 has 62.42, and on the valve 54.63, where MobileNetV2 has 72.77.
The limit
So the test, as registered before the run, is not established. The baselines' overall scores cover seven machine types, KELVIN's five.
The baselines' per-machine AUCs are as published on the challenge's results page, which does not state how they were aggregated; KELVIN's is the harmonic mean over both domains and three sections. So the comparison per machine is close, but not exactly like for like.
Data and licence
The data: DCASE 2022 Challenge Task 2 data (Hitachi, Ltd. and NTT Corporation), CC BY-NC-SA 4.0; results by Pelloran KELVIN, research use. No recording from the data is shown here.
Whom to talk to
Questions about this study, or a problem like it in your field: write to us.