Project summary

	Enriching case data with nutritional information to analyse the impact of consumption on health
Project details	In addition to the many well-documented dimensions of inequality (income, wealth, education, housing, access to healthcare and public services in general), disparities in food consumption are also likely to be a source of health inequalities, as well as social and territorial markers. Checkout data could provide a very rich description of local consumption, provided that product identifiers allow enrichment with external sources, such as nutritional information. The aim of this project is to enrich supermarket data with nutritional information extracted from Open Food Facts, supplemented by Ciqual data from Anses.
Players	Insee
Project results	To compensate for the partial matching via the barcode, a method for efficiently matching short labels was implemented. After a pre-processing stage to normalise short labels, fuzzy matching techniques are implemented. These are based on several tokenizers (including n-grams) by querying an ElasticSearch custom index and validating candidate echoes with a Levenstein distance. The pipeline consists of several stages successively relaxing constraints to find relevant candidates. Finally, the final match is evaluated using a similarity measure based on word embedding obtained by training a Siamese network on the exact match via barcodes. The data used, covering the period 2015-2018, is that of several chains belonging to the same mass retail group (relevanC).
Project products and documentation	- Enriching checkout data with nutritional information: a fuzzy matching approach on high-dimensional data, 2022 Statistical Methodology Days (Journées de méthodologie statistique 2022)

Similar projects

An assessment of cross-border tobacco purchases and associated tax losses in France

Methodological work on the Family Budget survey

automatic coding

data extraction

scanner data

Modernisation of the family budget survey using automatic classification tools

1 Jan 2022

Jocas, webscraping online job offers

The project Jocas (Job offers collection and analysis system) project enables the DARES (Ministerial Statistical Office for Labour) to automatically collect job offers…

1 Jan 2022

Automatic coding of companies’ main activity

Develop a machine learning algorithm to automate the classification of companies’ main activities and put it into production

1 Jan 2022

Modelling road vehicle fleet ownership and use

Matching administrative databases and modelling to estimate the number of road vehicles in France

1 Jun 2021

Comparison of matching methods and the contribution of machine learning

matching

administrative data

in production

To test and compare different matching methods in order to draw up recommendations for the work needed to build directories, particularly as part of the RESIL multiannual…

1 Jan 2021

Automatic coding of occupations in the PCS 2020 nomenclature

Automatically code occupations as part of the switch to the new PCS nomenclature (PCS 2020)

1 Jan 2021

Using credit card data and mobile phone data to forecast economic activity

The 2020 health crisis required a review of forecasting processes to be more responsive to events. INSEE used credit card transaction data to forecast economic activity.

1 Dec 2020

Population movements around the March 2020 containment using mobile phone network operators data

INSEE has had access to mobile telephony data as part of the monitoring of the 2020 health crisis. These data were used to produce the following statistics on population…

1 Nov 2020