BoolQ

Boolean Questions

Dataset Information
Modalities
Texts
Languages
English
Introduced
2019
License
Homepage

Overview

BoolQ is a question answering dataset for yes/no questions containing 15942 examples. These questions are naturally occurring – they are generated in unprompted and unconstrained settings.
Each example is a triplet of (question, passage, answer), with the title of the page as optional additional context.

Questions are gathered from anonymized, aggregated queries to the Google search engine. Queries that are likely to be yes/no questions are heuristically identified and questions are only kept if a Wikipedia page is returned as one of the first five results, in which case the question and Wikipedia page are given to a human annotator for further processing. Annotators label question/article pairs in a three-step process. First, they decide if the question is good, meaning it is comprehensible, unambiguous, and requesting factual information. This judgment is made before the annotator sees the Wikipedia page. Next, for good questions, annotators find a passage within the document that contains enough information to answer the question. Annotators can mark questions as “not answerable” if the Wikipedia article does not contain the requested information. Finally, annotators mark whether the question’s answer is “yes” or “no”. Only questions that were marked as having a yes/no answer are used, and each question is paired with the selected passage instead of the entire document.

Source: https://github.com/google-research-datasets/boolean-questions
Image Source: BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions

Variants: BoolQ

Associated Benchmarks

This dataset is used in 5 benchmarks:

Recent Benchmark Submissions

Task Model Paper Date
Question Answering Shakti-LLM (2.5B) SHAKTI: A 2.5 Billion Parameter … 2024-10-15
parameter-efficient fine-tuning LLaMA2-7b GIFT-SW: Gaussian noise Injected Fine-Tuning … 2024-08-27
Question Answering Mistral-Nemo 12B (HPT) Hierarchical Prompting Taxonomy: A Universal … 2024-06-18
Question Answering LLaMA3+MoSLoRA Mixture-of-Subspaces in Low-Rank Adaptation 2024-06-16
Classification OPT-125M Achieving Dimension-Free Communication in Federated … 2024-05-24
Classification OPT-1.3B Achieving Dimension-Free Communication in Federated … 2024-05-24
Question Answering LLaMA-2 13B + MixLoRA MixLoRA: Enhancing Large Language Models … 2024-04-22
Question Answering LLaMA-2 7B + MixLoRA MixLoRA: Enhancing Large Language Models … 2024-04-22
Question Answering LLaMA-3 8B + MixLoRA MixLoRA: Enhancing Large Language Models … 2024-04-22
parameter-efficient fine-tuning LLaMA2-7b DoRA: Weight-Decomposed Low-Rank Adaptation 2024-02-14
Question Answering LLaMA 2 34B (0-shot) Llama 2: Open Foundation and … 2023-07-18
Question Answering LLaMA 2 7B (zero-shot) Llama 2: Open Foundation and … 2023-07-18
Question Answering LLaMA 2 70B (0-shot) Llama 2: Open Foundation and … 2023-07-18
Question Answering LLaMA 2 13B (0-shot) Llama 2: Open Foundation and … 2023-07-18
parameter-efficient fine-tuning LLaMA2-7b QLoRA: Efficient Finetuning of Quantized … 2023-05-23
Question Answering PaLM 2-S (1-shot) PaLM 2 Technical Report 2023-05-17
Question Answering PaLM 2-M (1-shot) PaLM 2 Technical Report 2023-05-17
Question Answering PaLM 2-L (1-shot) PaLM 2 Technical Report 2023-05-17
Question Answering Bloomberg GPT 50B (1-shot) BloombergGPT: A Large Language Model … 2023-03-30
Question Answering BLOOM 176B (1-shot) BloombergGPT: A Large Language Model … 2023-03-30

Research Papers

Recent papers with results on this dataset: