Categories
Uncategorized

Federal Court of Appeal Bulk Decisions Dataset

Description: This is a bulk open-access dataset in JSON, parquet and Hugging Face dataset formats with the full text of Federal Court of Appeal (Canada) decisions. The process through which data is processed and code snippets for loading the data are available in a repository on the Refugee Law Lab GitHub.

Data: https://github.com/Refugee-Law-Lab/fca_bulk_data/tree/master/DATA/YEARLY

GitHub Repository: https://github.com/Refugee-Law-Lab/fca_bulk_data

Hugging Face Repository: https://huggingface.co/datasets/refugee-law-lab/canadian-legal-data

Current Coverage: 2001 – 2023 (* cases with neutral citation; to July 1 in 2023)

Number of Decisions: ~14,000

Languages: English & French

Format: JSON (yearly files), Parquet, Hugging Face Dataset

License: Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0). NOTE: Users must also comply with upstream licensing from the FCA data source, as well as requests on source urls not to allow indexing of the documents by search engines to protect privacy.

Citation: Sean Rehaag, “Federal Court of Appeal Bulk Decisions Dataset” (2023), online: Refugee Law Laboratory https://refugeelab.ca/bulk-data/fca

Data Fields:

  • citation1 (string): Legal citation for the document (neutral citation where available)
  • citation2 (string): For some documents multiple citations are available (e.g. for some periods the Supreme Court of Canada provided both official reported citation and neutral citation)
  • dataset (string): The name of the dataset (in this case “FCA”)
  • year (int32): Year of the document date, which can be useful for filtering
  • name (string): Name of the document, typically the style of cause of a case
  • language (string): Language of the document, “en” for English, “fr” for French, “” for no language specified
  • document_date (string): Date of the document, typically the date of a decision (yyyy-mm-dd)
  • source_url (string): URL where the document was scraped and where the official version can be found
  • scraped_timestamp (string): Date the document was scraped (yyyy-mm-dd)
  • unofficial_text (string): Full text of the document (unofficial version, for official version see source_url)
  • other (string): Field for additional metadata in JSON format, currently a blank string for most datasets

Programmatic Access in Python (via Hugging Face Datasets):

from datasets import load_dataset
import pandas as pd

dataset = load_dataset("refugee-law-lab/canadian-legal-data", "FCA", split="train")

# convert to dataframe
df = pd.DataFrame(dataset)
df

Programmatic Access in Python (via Parquet):

import pandas as pd
import requests
from io import BytesIO

url = 'https://huggingface.co/datasets/refugee-law-lab/canadian-legal-data/resolve/main/SCC/train.parquet'

# load data
results = requests.get(url)

# convert to dataframe
df = pd.read_parquet(BytesIO(results.content))
df

Programmatic Access in Python (via JSON):

import pandas as pd
import json
import requests

# Set variables
start_year = 1877  # First year of data sought (1877 +)
end_year = 2023  # Last year of data sought (2023 -)

# load data
base_ulr = 'https://raw.githubusercontent.com/Refugee-Law-Lab/scc_bulk_data/master/DATA/YEARLY/'
results = []
for year in range(start_year, end_year+1):
    url = base_ulr + f'{year}.json'
    results.extend(requests.get(url).json())

# convert to dataframe
df = pd.DataFrame(results)
df