lupa

lupa is an auditable R toolkit for profiling tabular data, defining what quality means for a particular use, measuring it, cleaning a copy without silently changing the input, and finding approximate duplicate records at scale. It reports the scope, evidence, and uncertainty of every result.

The public API, help pages, and vignettes are in Spanish. The names are stable and can be copied from this English guide; the Spanish README tells the same story in Spanish.

🌎 API language

The public names are Spanish in both examples and help pages:

Spanish API	English meaning
perfilar()	profile
analizar()	analyse
marco_calidad()	quality framework
planificar_limpieza()	plan a cleanup
guiar_limpieza()	guide a cleanup
aplicar()	apply a selected cleanup
medir() / evaluar()	measure / evaluate
detectar_duplicados_aproximados()	find approximate duplicates
reportar()	create a report

✨ What lupa does

• Profiles a delivery and surfaces missingness, types, patterns, dates, and personal-data evidence.
• Finds keys, relationships, dependencies, and measurement granularities that were never declared.
• Lets a project define its own quality framework instead of forcing a global score.
• Measures and evaluates explicit metrics, scales, validity rules, and referential domains.
• Produces editable cleanup plans, applies only selected actions to a copy, and keeps an audit log.
• Finds approximate duplicates with exact tiles, deterministic MinHash/LSH, blocking, cost estimates, and disk-backed lots.
• Repairs encoding damage in R, including repeated mojibake and CESU-8, while refusing unsafe lossy conversions.
• Follows quality through time and creates self-contained HTML reports.

📦 Installation

Until the first CRAN release, install the development version directly from GitHub:

# install.packages("pak")
pak::pak("sebollin/lupa")

For a local clone or release tarball, use:

R CMD build .
R CMD INSTALL lupa_0.1.0.tar.gz

Or install a local source tarball from R:

install.packages("lupa_0.1.0.tar.gz", repos = NULL)

⚡ A five-minute start

library(lupa)
data(datos_operativos)

perfil <- perfilar(datos_operativos, analizar_dependencias = FALSE)
head(perfil$hallazgos[, c("columna", "tipo_hallazgo", "severidad")], 5)

analisis <- analizar(datos_operativos)
archivo <- tempfile(fileext = ".html")
reportar(analisis, archivo = archivo)
stopifnot(file.exists(archivo))
unlink(archivo)

The profile is read-only: it never changes the input table. Findings are ordinary inspectable data frames, and personal-data evidence is masked when the classification warrants it. A real console preview is shown below.

A captured perfilar() console result

🧭 What can I do with lupa?

The pkgdown reference and the linked vignettes are the detailed manual. This table is the short map:

Task	Main functions	Read more
Look at data for the first time	perfilar(), analizar(), distribucion_valores(), detectar_asociaciones(), analizar_tiempo(), clasificar_variables(), inferir_tipo(), descubrir_patrones(), detectar_formatos_fecha(), sentinelas_naniar	Getting started
Find undeclared structure	detectar_claves(), detectar_relaciones(), detectar_dependencias(), granularidades(), transiciones_granularidad()	Getting started
Define quality	marco_calidad(), marco_agesic(), marco_iso25012(), catalogo_agesic(), metrica(), especializar(), instanciar(), modelo(), metricas_nucleo(), metricas_referencial(), proponer_modelo(), modelo_desde_propuesta(), perfiles_madurez(), cobertura_analisis()	Quality model
Measure and evaluate	medir(), agregar(), evaluar(), regla_evaluacion(), perfil_evaluacion(), escala(), referencial(), vigencia()	Quality model
Clean safely	planificar_limpieza(), guiar_limpieza(), aplicar()	Cleaning plan
Find approximate duplicates	detectar_duplicados_aproximados(), estimar_costo()	Scale and duplicates
Repair encoding damage	reparar_codificacion through planificar_limpieza() and aplicar()	Cleanup reference
Follow quality over time	historico_calidad(), acumular_historico(), guardar_historico(), leer_historico(), detectar_deriva_calidad(), comparar_perfiles(), comparar_evaluaciones()	History and drift
Share results	reportar(), guardar_analisis(), leer_analisis()	Reporting reference
Validate and extend	validadores_internacionales(), validadores_uruguay(), pack_validadores(), validar_ci_uy(), validar_rut_uy(), validar_luhn(), validar_mod97(), validar_iso3166(), validar_iso4217(), validar_correo()	Reference

library(lupa)
data(datos_operativos)
marco <- marco_calidad(
  "Marco operativo",
  list(Estructura = c("Ausencias observadas", "Duplicacion exacta"))
)
propuesta <- proponer_modelo(perfilar(datos_operativos,
                                      analizar_dependencias = FALSE))
list(marco = marco, propuesta = propuesta)

The API has a few boundaries worth knowing. There is no global quality score: dimensions, units, and rules stay visible. The core is universal and catalogues are pluggable; AGESIC v1.6 is a reference implementation, not a country lock. The package has one required import, cli; stringdist is optional.

🔍 Where it fits

skimr and DataExplorer explore; pointblank, validate, and dataquieR express or evaluate rules; zoomerjoin, textreuse, and reclin2 focus on text comparison or record linkage. calidad, maintained by Klaus Lehmann and Ricardo Pizarro, is a complementary axis: it evaluates the quality of survey estimates, while lupa evaluates the tabular data that produces an estimate.

Encoding repair follows the approach and frozen data of ftfy 6.3.1 by Robyn Speer, in R. It includes eleven byte tables, CESU-8 and Java C0 80 handling, and five deliberate extensions documented in the NEWS. It reproduces 159 of the 161 distributed corpus cases and leaves all 31 negative cases untouched. It deliberately does not provide ftfy’s style-oriented fix_text steps such as HTML unescaping, quote curling, width normalization, or Unicode normalization: changing legitimate data silently is not repair.

📖 Citation and references

citation("lupa")

Conceptual references are Batini and Scannapieco (2016), the AGESIC Digital Government Data Quality Framework v1.6, and ISO/IEC 25012:2008.

🤝 Contribute and report

Please use the issue tracker for bugs, proposals, and documentation fixes. The stable contracts are the declared units, scope, protection, and audit trail; implementation details and benchmark times can change between releases when those contracts remain true.

📄 License

lupa is released under the GPL-3. See LICENSE.note for the Apache-2.0 data derived from ftfy and the MIT data derived from naniar.