ELAN Tutorial 1: Set Up Tiers and Annotate Sign Language Data

Successful controlled vocabulary implementation requires strategic planning before data entry. Terms reside in separate tables and connect to multiple records, preventing redundant data entry. Best practice involves building complete term sets before creating individual records. Industries offer pre-made standards like BC Thesaurus, Nomenclature 3.0, LGMA, Library of Congress, and OLC records. However, customization is necessary for institutional specificity. Critical considerations include hierarchical structure for contextual understanding, formatting standards adherence, and treating authorities as complete bodies of work rather than loosely connected terms to prevent ad hoc management leading to disorganization.

Building controlled vocabularies requires a systematic methodological approach that begins with understanding the universe of knowledge within a specific context, establishing an ontological commitment to define how concepts relate, and then applying analytical-synthetic classification methods to transform word lists into structured taxonomies and associative thesauri that enable precise information retrieval.

Controlled vocabularies improve metadata consistency and reduce errors through standardized term selection. Implementation strategies include uploading CSV files containing vocabulary terms, allowing institutions to customize vocabularies by language or scope. Users select applicable vocabularies for specific metadata elements, then type partial terms to receive suggestions. For large networks spanning multiple locations, hierarchical vocabularies may be flattened for simplicity. This approach reduces arbitrary variations and misspellings while accommodating diverse institutional needs. Integration with existing vocabularies like AgroVoc provides extensive terminology coverage while maintaining flexibility for local customization.

Control vocabularies should be developed atomically and coordinated through a data office that drives consistency across vocabularies. Local CVs are designed for narrow use cases, while foundational CVs mature and are used by multiple groups. Collections and faceted CVs allow slicing through multiple vocabularies for specific purposes. Each vocabulary should include preferred labels, synonyms, language tags, alt labels for spellings, and schema references. This enables translation between different schemas and provides a robust foundation for data interoperability.

Controlled vocabularies solve metadata inconsistency by defining standardized terms for machine-readable processing. Without them, variations in capitalization, spelling, or language cause machines to treat equivalent concepts as distinct. CORE controlled vocabularies provide community-developed standards for resource types, access rights, and version types, available in multiple languages. OAIS guidelines specify implementation requirements, enabling repositories to use dropdown lists for human-friendly submission while storing standardized codes. This dual approach ensures both user accessibility and machine interoperability for global discovery.

This tutorial demonstrates how to export data from ELAN 5.9, a linguistic annotation software developed by the Max Planck Institute for Psycholinguistics. The video covers four main export functions: exporting search results to a tabulated text file (requiring the 'Include file name column' option), exporting transcriptions to tab-separated text format (with options to select specific lines or restrict to time intervals), exporting to interlinear text format (commonly used in linguistic publications, with options to show/hide line labels and restrict to selections), and exporting to subtitle files (requiring the file to be saved in the same folder as the video with a similar name for automatic recognition). The instructor emphasizes that exported files can be opened in Notepad and copied to Excel for further analysis.

ELAN is a linguistic analysis software used for transcribing and annotating audio and video recordings, featuring tools for creating tiers (transcription lines) and types (data categories), segmenting audio, and exporting data to formats like Word and Excel for both qualitative and quantitative linguistic research analysis.

ELAN is a standalone Java application for manual annotation of audio and video recordings, supporting multi-tier, multi-participant, time-linked annotations. Developed since 2000 at MPI for Psycholinguistics, it produces EAF XML files and runs on Windows, Mac, and Linux. Core features include unlimited annotation tiers with parent-child relationships, four inter-tier relations (time subdivision, symbolic subdivision, included in, symbolic association), and four working modes: annotation, segmentation, transcription, and interlinearization. Automatic annotation via recognizers handles speech/non-speech detection, boundary detection, and forced alignment. The software supports importing/exporting from Toolbox, Praat, and other formats, with quantitative analysis features including modified Kappa for inter-rater reliability.

This section covers exporting annotation data, adding calculations and summations, and creating final documentation. Users learn to export annotation data to Excel, add formulas for calculations, and document totals and quantities. The instructor demonstrates export procedures, calculation features, and documentation formats. Proper documentation ensures accurate communication of reinforcement requirements for fabrication and construction.

Software provides multiple export options for disseminating qualitative findings and extracting supporting evidence. Reports generate comprehensive documents showing sources, coding summaries, property distributions, canvas views, and quotes, customizable by detail level. Word clouds visualize language patterns with proportional sizing and customizable filters. Word documents export complete transcripts with color-coded annotations showing coding assignments. Spreadsheet exports (CSV) transform data into tabular format compatible with statistical software for quantitative analysis. Project export options include exporting sources only (preserving materials without codes) and structure only (preserving framework without text), supporting collaborative analysis and standardized evaluation criteria application across multiple projects.

ELAN software provides a search function that enables researchers to efficiently search across multiple annotated Elan files simultaneously, eliminating the need to manually examine each file individually; this is particularly useful for analyzing large corpora such as the Corpus NGT, which contains over 2000 annotated Elan files from Sign Language of the Netherlands data developed by Prof. Onno Crasborn's team at Radboud University Nijmegen.

This section covers two essential export functions in ELAN version 5.9. First, exporting search results involves performing a search, clicking the '6 hits' button, ensuring 'Include file name column' is checked, selecting a destination folder, and saving the file. The resulting file can be opened in Notepad and copied to Excel for analysis. Second, exporting transcriptions to tabulated text format requires selecting desired lines in the export window, specifying a filename and folder, and saving the file. For exporting only selected portions, users must first select the text segment and check 'Restrict to selected time interval' before proceeding.

ELAN is a standalone desktop application for manual annotation of audio and/or video recordings. It supports multi-layered or multi-tiered time-linked annotation, allowing users to create any number of tiers (layers) necessary to capture all phenomena of interest. Annotations are grouped into tiers that can be interrelated. The data produced by ELAN is stored in an XML file format called EAF (with .eaf file extension). Most of ELAN is written in Java programming language and is platform-independent, though important parts are platform-specific. The software is open-source with sources available for download.

ELAN is a standalone Java application for manual annotation of audio and video recordings, supporting multi-tier, multi-participant, time-linked annotations. Developed since 2000 at MPI for Psycholinguistics, it produces EAF XML files and runs on Windows, Mac, and Linux. Core features include unlimited annotation tiers with parent-child relationships, four inter-tier relations (time subdivision, symbolic subdivision, included in, symbolic association), and four working modes: annotation, segmentation, transcription, and interlinearization. Automatic annotation via recognizers handles speech/non-speech detection, boundary detection, and forced alignment. The software supports importing/exporting from Toolbox, Praat, and other formats, with quantitative analysis features including modified Kappa for inter-rater reliability.

Elan's structured search function provides three search modes: substring search (searches all annotation layers including explanations), single layer search (searches specific layers with case sensitivity options), and multiple layers search. The frequency view feature enables researchers to analyze annotation distributions, such as identifying the most common variant of a lexical sign or determining which classifier type (entity classifiers EC or handling classifiers HC) appears most frequently in a specific tier. Regular expression search allows tier-specific pattern matching for precise annotation analysis.

ELAN supports both qualitative and quantitative analysis including basic statistics generation and export for annotations, engram analysis for custom pattern extraction, structured search capabilities, inter-rater reliability calculation, annotation overlap and gap information, and interoperability with other tools through standardized formats.

Inter-rater reliability measures the degree of agreement among different annotators labeling the same data. When comparing annotations from two raters, a table can show where they agree and disagree. For instance, if both annotators consistently label 'bot challenge' but have confusion about 'correct orders,' this suggests the 'correct order' label may need clarification, splitting into separate labels, or removal. Such analysis helps improve annotation guidelines and label definitions.

This section addresses the challenges of working with human-annotated data. When working with human behavior data (Twitter messages, product reviews, tissue samples), annotations often suffer from subjectivity and inter-annotator disagreement. Different experts may have different opinions on the same data point. Researchers need to assess inter-rater agreement to ensure high-quality data for training classifiers. For segmented data (individual tweets), agreement can be calculated directly. For continuous data (time series), data must be discretized into segments before calculating agreement. This reliability assessment helps identify problematic data points and ensures consistent expert training.

This section addresses the critical need to measure assessor agreement since relevance assessment involves subjective decisions. Three approaches exist: probability of same decisions, statistical correlation, and percentage agreement. Kappa statistics are preferred as they account for random agreement. The formula κ = (p_o - p_e) / (1 - p_e) calculates observed minus expected agreement relative to maximum possible. Binary decisions use contingency matrices counting agreements and disagreements. Expected agreement combines probabilities of independent yes/no responses.

Additional analysis examined consistency among annotators by having 290 workers label 2,000 unique interface elements across 344 screenshots, with each element labeled five times. Results showed that only 58% of elements received identical labels across all workers. Agreement scores (Kappa statistic) indicated moderate inter-rater reliability at 0.50. This variability in human perception means that achieving perfect predictive models may be difficult, as human judgment itself is inconsistent. The model's predictions correlate well with this observed uncertainty, producing more definitive predictions when workers agree and showing higher uncertainty when annotations disagree.