Validation of an algorithm for automated classification of migraine and tension- type headache attacks in an electronic headache diary

DOI: https://doi.org/10.1186/s10194- 020- 01139- w

Abstract-Summary This study evaluates the accuracy of an automated classification tool of single attacks of the two major primary headache disorders migraine and tension-type headache used in an electronic headache diary.

One hundred two randomly selected reported headache attacks from an elec- tronic headache-diary of patients using the medical app M-sense were classified by both a neurologist with specialisation in headache medicine and an algo- rithm, constructed based on the ICHD-3 criteria for migraine and tension-type headache.

The level of agreement between the headache specialist and the algorithm was

compared by using a kappa statistic.

The neurologist and the algorithm classified migraines with aura (MA), migraines without aura (MO), tension-type headaches (TTH) and non-migraine or non- TTH events.

Of the 102 headache reports, 86 cases were fully agreed on, and 16 cases not, making the level of agreement unweighted kappa 0.74 and representing a substan- tial level of agreement.

470

3 Diagnosis

The substantial level of agreement indicates that the classification tool is a valu- able instrument for automated evaluation of electronic headache diaries, which can thereby support the diagnostic and therapeutic clinical processes.

Extended: The level of agreement between the neurologist and the algorithm’s classification of 102 single headache events resulted in 86 cases of agreement and 16 cases of disagreement.

The findings from this validity assessment in the below section of results. Future research can use this classification algorithm for large scale database analysis for epidemiological studies, for example to investigate whether migraine and tension-type headache are diagnostic types or points on a severity contin- uum [62].

Background Regarding treatments, patients with high severity of migraine and headache-related disability should receive acute and, if necessary, preventive migraine-specific ther- apy [63].

To resolve this need, in this paper, we present an algorithm that applies the ICHD-3 criteria to single headache events recorded in a migraine management app’s database.

Our goal is to provide an efficient means to classify patient headache events as

migraine or tension-type headache.

The aim being to investigate how accurately an algorithm classifies patient head- ache events as migraine or tension-type headache in electronic health diaries using ICHD criteria.

As validation, both a neurologist specialised in headache medicine and the algo-

rithm classified the headache-diary data from a medical apps’ database.

Patients use this medical app for documenting headaches as well as potential

trigger factors, all of which get summarized in reports for doctors.

Methods We developed an algorithm to classify primary headache disorders according to ICHD-3 criteria for both definite and probable Migraine without Aura, Migraine with Aura, and TTH as for usage in the M-sense app.

In the first phase, a computer-based algorithm based on ICHD-3 criteria was run

and classified the 102 single headache events taken from the M-sense database.

Of the validation study, the headache specialist classified the same 102 headache events also according to the criteria of ICHD-3 with information about an existing diagnosis of migraine and tension-type headache.

Based on the evaluation using the headache sheet, the neurologist assigned the classification of migraine without aura (MO), migraine with aura (MA), TTH, or non-migraine or non-TTH (non-classifiable).

We calculated the kappa statistic to compare the algorithm’s classification results to each of the neurologist’s classifications based on the single-entry headache sheets.

3.1 Classification

471

Results The level of agreement between the neurologist and the algorithm’s classification of 102 single headache events resulted in 86 cases of agreement and 16 cases of disagreement.

From the neurologist’s answers to the short questionnaire, we deduced that the

algorithm correctly applied the ICHD-3 criteria in the 11 cases of category 1–4.

For subcategory five, in which the neurologist had categorized four cases as non- classifiable in contrast to the algorithm’s identification as MO or TTH, three of four of these cases had a short headache duration < 30 min in common. For the other case, the neurologist corrected his classification. For subcategory six, wherein the neurologist identified a case to be migraine without aura and the algorithm non-classifiable, we found that the algorithm was not coded to interpret the relevant ICHD criteria correctly.

Discussion Results from the current study demonstrate that the investigated algorithm for iden- tifying headaches is a valid instrument for automated evaluation of electronic head- ache diaries.

This result is not surprising, given that the evaluation of a whole headache diary by classifying large numbers of individual attacks is a tedious task that requires high levels of concentration and does not reflect common clinical practice in headache diagnosis.

One study identified that agreement between neurologists asked to assign a head- ache diagnosis based on the review of videotaped patient interviews, ranged in a kappa from 0.55 to 0.81 [64].

Since migraine and TTH themselves are phenomenological diagnoses, other pos- sible diagnoses, such as secondary headaches, must be excluded via differential diagnosis which is reflected by the criterion E in ICHD-3.

Criterion A in ICHD-3 defines the number of attacks or headache days that are

necessary before a diagnosis can be made [65].

Conclusion The results of this study confirm the accuracy of an algorithm for automated clas- sification of MA, MO, and TTH, with a substantial level of agreement to a neurolo- gist specialized in headache medicine.

Future research can use this classification algorithm for large scale database analysis for epidemiological studies, for example to investigate whether migraine and tension-type headache are diagnostic types or points on a severity contin- uum [62].

Acknowledgement A machine generated summary based on the work of Roesch, Aaron; Dahlem, Markus A; Neeb, Lars; Kurth, Tobias. 2020 in The Journal of Headache and Pain.

472

3 Diagnosis

Classifying migraine subtypes and their characteristics by latent class analysis using data of a nation-wide population-based study