Our method

Six steps, run on the project’s computers, from the channels’ feeds to the published record. Each step opens on its detail: models, thresholds, reliability.

The methods paper (PDF)

  1. 01

    Detect

    The channels’ feeds are read every six hours, their full list every week. The videos of the last seven days go ahead of everything else.

    The detail
  2. 02

    Extract

    The video, its metadata and its comments are extracted. The audience is measured again at day 1, 7 and 30, then every month.

    The detail
  3. 03

    Transcribe

    The voice is isolated from the music, speakers told apart, speech transcribed in the language spoken, then cut into sentences.

    The detail
  4. 04

    Classify

    Each sentence is classified as political or not, then assigned to nine themes by nine classifiers whose reliability is published.

    The detail
  5. 05

    The people named

    The public figures, parties and media of a closed list are recognised in the sentences and counted video by video.

    The detail
  6. 06

    Summarise

    A local language model writes the record of each video of the week, checked against the transcript, then the synthesis and the arguments.

    The detail
Around the steps

The corpus, the arguments, the removals, the limits

The corpus Three countries, seven orientations, the channels chosen and coded by the team. The eleven steps The full diagram, from the task queue to publication, step by step. Arguments by theme A model reads a sample of each theme’s sentences and draws out the arguments. Vanished videos The removal is noticed, dated, explained when the platform says why; the record stays. Coverage and limits What is transcribed, what is not yet, the known limits, the corrections log.
Project paper