Datasets Resource Center

Created and curated for teams working on world-class AI applications

Off-the-Shelf Datasets

Our high-quality licensable datasets to jumpstart your AI projects

We offer an extensive catalog of ‘Off-the-Shelf’ datasets, with over 250 licensable datasets comprising of over 11,000 hours of audio, over 25,000 images and over 8.7 million words across 80 languages and multiple dialects. Our Off-the-Shelf datasets are designed to effectively improve accuracy, overall performance and to quickly deliver high quality datasets at scale for specific AI program needs. Among our offerings, you will find datasets across multiple data types, including image, video, speech, audio, and text. We are constantly building new datasets to meet the needs of our global customer base.

Open-Source Public Datasets

Curated recommendations from our data scientists for your Al projects

Machine Learning and Artificial Intelligence applications require significant amounts of data to train. You can search for open datasets to access, modify, reuse, and share, from our recommended resources. Use these publicly available datasets to influence the development of AI and ML applications or if you want a simple dataset to benchmark a solution or compare different algorithms before tackling a real dataset. These open datasets are a great option to consider for access to data that lies outside the scope of your organization.

Dataset Finders

Use Kaggle to find data sets, explore and build models and work with other data scientists and Machine Learning engineers. Explore and analyze a collection of over 50,000 public datasets on everything from bone x-rays to results from boxing bouts.

Explore over 500 data sets of the Machine Learning Repository from UC Irvine, through a searchable interface. Datasets range across many topics, vary in terms of size, from only a few cases (or “instances”) up to over 43 million and from only 1 or 2 variables (or “attributes”) to over a million variables.

Computer Vision

Computer vision enables computers to identify and process objects in images and videos in the same way that humans do, by emulating parts of the complexity of the human vision system. Leverage Machine Learning for image applications such as enabling self-driving cars to make sense of their surroundings, facial recognition applications, augmented and mixed reality or automate tasks finding symptoms in x-ray and MRI scans in healthcare. Build a robust Computer Vision model using a rich collection of Computer Vision datasets.

Accelerate AI development using 1000+ High-quality Open Datasets. Choose from 50+ application scenarios, 30+ annotation types, and 10+ data formats.

These datasets include diverse topics from recognizing objects to reconstructing a 3D room, from finding a person in a video to identifying a shirt in a photo. The datasets can be sorted by published date or topic, and users can search with keywords to locate images appropriate to their needs.

Use these open datasets to build facial recognition applications, virtual reality gadgets, sensory detection, holographic imaging and much more.

More than 3,000 Machine Learning Datasets. Find datasets by task and modality, compare usage over time, browse benchmarks and more.

Open-source datasets for Computer Vision Machine Learning models across a wide array of domains- animals, board games, self-driving cars, medicine, thermal imagery, aerial drone images, and even synthetically generated data. You can freely download images and annotations in any format: VOC XML, COCO JSON, YOLOv3 flat text files, even TFRecords.

Speech Corpora

Recording and transcribing new Speech Corpora to create acoustic models and train Speech Recognition engines can be time consuming and expensive. Use open databases of speech audio files and text transcriptions to quickly and cheaply building transcribed Speech orpora containing utterances from many speakers in a variety of acoustic conditions.

A central place for speech resources, OpenSLR hosts speech and language resources, such as training Corpora for Speech Recognition, and software related to Speech Recognition.

Candlewill

A collection of Speech Corpus for Automatic Speech Recognition (ASR) and Text-To-Speech (TTS).

Edresson

The dataset has 71,358 total number of words, with 13,311 distinct words, approximately 10 hours and 28 minutes of speech from a single speaker, recorded at 48Khz, containing a total of 3,632 audio files in Wave format. Audio files range from 0.67 to 50.08 seconds.

Designed to provide speech data for the acquisition of acoustic-phonetic knowledge and for the development and evaluation of Automatic Speech Recognition systems. Contains a total of 6300 sentences, 10 sentences spoken by each of 630 speakers from 8 major dialect regions of the United States.

VoxCeleb

Audio-visual dataset consisting of short clips of human speech, extracted from interview videos uploaded to YouTube. This dataset contains 7,000 + speakers, 1 million + utterances and 2,000 + hours of both audio and video.

msang

A Twitter Corpus built with the aim of representing and analyzing hate speech against some minority groups in Italy: immigrants in particular, but also Muslims and Roma. The contains the tweets' ID and their annotation.

VoxForge

Transcribed speech for use in Speech Recognition Engines; categorize and make available all submitted audio files (Speech Corpus) and Acoustic Models.

homink

A Korean read Speech Corpus of about 120 hours from the National Institute of Korean Language (NIKL).

siddiquelatif

URDU dataset contains emotional utterances of Urdu speech gathered from Urdu talk shows. It contains 400 utterances of four basic emotions: Angry, Happy, Neutral, and Emotion. There are 38 speakers (27 male and 11 female).

Common Voice dataset, an open-source dataset of voices, currently consists of over 7,000 validated hours in 60 languages and includes demographic metadata like age, sex, and accent that can help train the accuracy of Speech Recognition engines. Each entry in the dataset consists of a unique MP3 and corresponding text file.

A large database of sentences and translations to see examples of how words are used in the context of a sentence.

Made from audio talks and their transcriptions, the dataset contains 1495 audio talks in NIST sphere format (SPH), 1495 transcripts in STM format, dictionary with pronunciation (159, 848 entries) and selected monolingual data for language modeling.

Data Collection

If a more customized data set is needed for your specific use case, we provide data collection as a standalone service as well as a part of a multi-component deliverable such as an ASR speech database that typically includes audio data, transcription, pronunciation lexicon, and a language-specific document or an annotated image dataset. Our data collection services span a variety of data types and collection methodologies for a range of environments to best meet your unique data requirements.

Latest News and Resources

to launch ai be prepared to scale

to launch ai be prepared to scale

<picture class="attachment-recommended size-recommended wp-post-image"> <source type=image/webp srcset=https://resources.appen.com/wp-content/uploads/2021/04/To-Launch-AI-Successfully-Be-Prepared-to-Scale-330x196.png.webp> <img width=330 height=196 src=https://resources.appen.com/wp-content/uploads/2021/04/To-Launch-AI-Successfully-Be-Prepared-to-Scale-330x196.png alt="to launch ai be prepared to scale"> </picture>

To Launch AI Successfully, Be Prepared to Scale

AI Data Acquisition and Governance

AI Data Acquisition and Governance

<picture class="attachment-recommended size-recommended wp-post-image"> <source type=image/webp srcset=https://resources.appen.com/wp-content/uploads/2021/04/AI-Data-Acquisition-and-Governance-330x196.png.webp> <img width=330 height=196 src=https://resources.appen.com/wp-content/uploads/2021/04/AI-Data-Acquisition-and-Governance-330x196.png alt="AI Data Acquisition and Governance"> </picture>

AI Data Acquisition and Governance

Four Tips to Pick Your Goldilocks AI Project

Four Tips to Pick Your Goldilocks AI Project

<picture class="attachment-recommended size-recommended wp-post-image"> <source type=image/webp srcset=https://resources.appen.com/wp-content/uploads/2021/03/Four-Tips-to-Pick-Your-Goldilocks-AI-Project-330x196.png.webp> <img width=330 height=196 src=https://resources.appen.com/wp-content/uploads/2021/03/Four-Tips-to-Pick-Your-Goldilocks-AI-Project-330x196.png alt="Four Tips to Pick Your Goldilocks AI Project"> </picture>

Four Tips to Pick Your Goldilocks Problem for AI

Active Learning vs Weak Supervision

Active Learning vs Weak Supervision

<picture class="attachment-recommended size-recommended wp-post-image"> <source type=image/webp srcset=https://resources.appen.com/wp-content/uploads/2021/03/Active-Learning-vs-Weak-Supervision-330x196.png.webp> <img width=330 height=196 src=https://resources.appen.com/wp-content/uploads/2021/03/Active-Learning-vs-Weak-Supervision-330x196.png alt="Active Learning vs Weak Supervision"> </picture>

ML Techniques: Active Learning vs Weak Supervision

London School of Economics Case Study

London School of Economics Case Study

<picture class="attachment-recommended size-recommended wp-post-image"> <source type=image/webp srcset=https://resources.appen.com/wp-content/uploads/2021/03/London-School-of-Economics-Case-Study-330x196.png.webp> <img width=330 height=196 src=https://resources.appen.com/wp-content/uploads/2021/03/London-School-of-Economics-Case-Study-330x196.png alt="London School of Economics Case Study"> </picture>

London School of Economics Takes Agile Approach to Data Labeling with Appen

Real World AI Now Available

Real World AI Now Available

<picture class="attachment-recommended size-recommended wp-post-image"> <source type=image/webp srcset=https://resources.appen.com/wp-content/uploads/2021/03/Real-World-AI-330x196.png.webp> <img width=330 height=196 src=https://resources.appen.com/wp-content/uploads/2021/03/Real-World-AI-330x196.png alt="Real World AI Now Available"> </picture>

AI Experts Provide Comprehensive Insights in Real World AI: A Practical Guide for Responsible Machine Learning

what is video annotation

what is video annotation

<picture class="attachment-recommended size-recommended wp-post-image"> <source type=image/webp srcset=https://resources.appen.com/wp-content/uploads/2021/03/what-is-video-annotation-330x196.png.webp> <img width=330 height=196 src=https://resources.appen.com/wp-content/uploads/2021/03/what-is-video-annotation-330x196.png alt="what is video annotation"> </picture>

Video Annotation: What Is It and How Automation Can Help

callminer case study

callminer case study

<picture class="attachment-recommended size-recommended wp-post-image"> <source type=image/webp srcset=https://resources.appen.com/wp-content/uploads/2021/03/callminer-case-study-330x196.png.webp> <img width=330 height=196 src=https://resources.appen.com/wp-content/uploads/2021/03/callminer-case-study-330x196.png alt="callminer case study"> </picture>

CallMiner Delivers Fast and Accurate Customer Insights with Large-Scale Annotation Solution

computer vision data sheet

computer vision data sheet

<picture class="attachment-recommended size-recommended wp-post-image"> <source type=image/webp srcset=https://resources.appen.com/wp-content/uploads/2021/03/computer-vision-data-sheet-330x196.png.webp> <img width=330 height=196 src=https://resources.appen.com/wp-content/uploads/2021/03/computer-vision-data-sheet-330x196.png alt="computer vision data sheet"> </picture>

Appen's Computer Vision Expertise

Natural Language Processing & Speech Technology data sheet

Natural Language Processing & Speech Technology data sheet

<picture class="attachment-recommended size-recommended wp-post-image"> <source type=image/webp srcset=https://resources.appen.com/wp-content/uploads/2021/03/Natural-Language-Processing-Speech-Technology-data-sheet-330x196.png.webp> <img width=330 height=196 src=https://resources.appen.com/wp-content/uploads/2021/03/Natural-Language-Processing-Speech-Technology-data-sheet-330x196.png alt="Natural Language Processing & Speech Technology data sheet"> </picture>

Natural Language Processing & Speech Technology at Appen

model maintenance

model maintenance

<picture class="attachment-recommended size-recommended wp-post-image"> <source type=image/webp srcset=https://resources.appen.com/wp-content/uploads/2021/02/model-maintenance-330x196.png.webp> <img width=330 height=196 src=https://resources.appen.com/wp-content/uploads/2021/02/model-maintenance-330x196.png alt="model maintenance"> </picture>

AI Model Maintenance: A Guide to Managing a Model Post-Production

New Off-the-Shelf (OTS) Datasets

<img width=330 height=196 src=https://resources.appen.com/wp-content/uploads/2021/03/abstract-crowds-of-people-with-virtual-reality-street-display-picture-id1149151631-330x196.jpg class="attachment-recommended size-recommended wp-post-image" alt="New Off-the-Shelf (OTS) Datasets">

New Off-the-Shelf (OTS) Datasets from Appen Accelerate AI Deployment

AI-Powered Search Relevance Machine Learning

AI-Powered Search Relevance Machine Learning

<picture class="attachment-recommended size-recommended wp-post-image"> <source type=image/webp srcset=https://resources.appen.com/wp-content/uploads/2021/02/Search-Relevance-Machine-Learning-330x196.png.webp> <img width=330 height=196 src=https://resources.appen.com/wp-content/uploads/2021/02/Search-Relevance-Machine-Learning-330x196.png alt="AI-Powered Search Relevance Machine Learning"> </picture>

What is AI-Powered Search Relevance?

What is image annotation

What is image annotation

<picture class="attachment-recommended size-recommended wp-post-image"> <source type=image/webp srcset=https://resources.appen.com/wp-content/uploads/2021/02/What-is-image-annotation-330x196.png.webp> <img width=330 height=196 src=https://resources.appen.com/wp-content/uploads/2021/02/What-is-image-annotation-330x196.png alt="What is image annotation"> </picture>

What Is Image Annotation and How Is It Used To Build AI Models?

QA for Autonomous Vehicle Manufacturers

<img width=330 height=196 src=https://resources.appen.com/wp-content/uploads/2020/08/self-driving-smart-vehicle-picture-id1077892268-330x196.jpg class="attachment-recommended size-recommended wp-post-image" alt="QA for Autonomous Vehicle Manufacturers">

Appen Delivers High-Quality Training Data and Quality Assurance Services for Autonomous Vehicle Manufacturers

View All Resources

Website for deploying AI with world class training data