498 lines
28 KiB
HTML
498 lines
28 KiB
HTML
<h1 id="awesome-audit-algorithms-awesome">Awesome Audit Algorithms <a
|
||
href="https://awesome.re"><img src="https://awesome.re/badge-flat.svg"
|
||
alt="Awesome" /></a></h1>
|
||
<p>A curated list of algorithms for auditing black-box algorithms.
|
||
Nowadays, many algorithms (recommendation, scoring, classification) are
|
||
operated at third party providers, without users or institutions having
|
||
any insights on how they operate on their data. Audit algorithms in this
|
||
list thus apply to this setup, coined the “black-box” setup, where one
|
||
auditor wants to get some insight on these remote algorithms.</p>
|
||
<p><img src="https://github.com/erwanlemerrer/awesome-audit-algorithms/blob/main/resources/audit.png" width="600" alt="banner" class="center"></p>
|
||
<blockquote>
|
||
<p>A user queries a remote algorithm (eg, through available APIs), to
|
||
infer information about that algorithm.</p>
|
||
</blockquote>
|
||
<h2 id="contents">Contents</h2>
|
||
<ul>
|
||
<li><a href="#papers">Papers</a></li>
|
||
<li><a href="#related-events">Related Events
|
||
(conferences/workshops)</a></li>
|
||
</ul>
|
||
<h2 id="papers">Papers</h2>
|
||
<h3 id="section">2025</h3>
|
||
<ul>
|
||
<li><a href="https://arxiv.org/abs/2504.00874">P2NIA: Privacy-Preserving
|
||
Non-Iterative Auditing</a> - (ECAI) <em>Proposes a mutually beneficial
|
||
collaboration for both the auditor and the platform: a
|
||
privacy-preserving and non-iterative audit scheme that enhances fairness
|
||
assessments using synthetic or local data, avoiding the challenges
|
||
associated with traditional API-based audits.</em></li>
|
||
<li><a
|
||
href="https://www.cambridge.org/core/services/aop-cambridge-core/content/view/9E8408C67F7CE30505122DD1586D9FA2/S3033373325000080a.pdf/the-fair-game-auditing-and-debiasing-ai-algorithms-over-time.pdf">The
|
||
Fair Game: Auditing & debiasing AI algorithms overtime</a> -
|
||
(Cambridge Forum on AI: Law and Governance) <em>Aims to simulate the
|
||
evolution of ethical and legal frameworks in the society by creating an
|
||
auditor which sends feedback to a debiasing algorithm deployed around an
|
||
ML system.</em></li>
|
||
<li><a href="https://arxiv.org/pdf/2505.04796">Robust ML Auditing using
|
||
Prior Knowledge</a> - (ICML) <em>Formally establishes the conditions
|
||
under which an auditor can prevent audit manipulations using prior
|
||
knowledge about the ground truth.</em></li>
|
||
<li><a href="https://arxiv.org/abs/2501.02997">CALM: Curiosity-Driven
|
||
Auditing for Large Language Models</a> - (AAAI) <em>Auditing as a
|
||
black-box optimization problem where the goal is to automatically
|
||
uncover input-output pairs of the target LLMs that exhibit illegal,
|
||
immoral, or unsafe behaviors.</em></li>
|
||
<li><a href="https://arxiv.org/abs/2412.13021">Queries, Representation
|
||
& Detection: The Next 100 Model Fingerprinting Schemes</a> - (AAAI)
|
||
<em>Divides model fingerprinting into three core components, to identify
|
||
∼100 previously unexplored combinations of these and gain insights into
|
||
their performance.</em></li>
|
||
</ul>
|
||
<h3 id="section-1">2024</h3>
|
||
<ul>
|
||
<li><a href="https://arxiv.org/pdf/2411.05197">Hardware and software
|
||
platform inference</a> - (arXiv) <em>A method for identifying the
|
||
underlying GPU architecture and software stack of a black-box machine
|
||
learning model solely based on its input-output behavior.</em></li>
|
||
<li><a href="https://arxiv.org/abs/2407.13281">Auditing Local
|
||
Explanations is Hard</a> - (NeurIPS) <em>Gives the (prohibitive) query
|
||
complexity of auditing explanations.</em></li>
|
||
<li><a href="https://arxiv.org/abs/2409.00159">LLMs hallucinate graphs
|
||
too: a structural perspective</a> - (complex networks) <em>Queries LLMs
|
||
for known graphs and studies topological hallucinations. Proposes a
|
||
structural hallucination rank.</em></li>
|
||
<li><a href="https://arxiv.org/pdf/2402.08522">Fairness Auditing with
|
||
Multi-Agent Collaboration</a> - (ECAI) <em>Considers multiple agents
|
||
working together, each auditing the same platform for different
|
||
tasks.</em></li>
|
||
<li><a href="https://arxiv.org/pdf/2401.11194">Mapping the Field of
|
||
Algorithm Auditing: A Systematic Literature Review Identifying Research
|
||
Trends, Linguistic and Geographical Disparities</a> - (Arxiv)
|
||
<em>Systematic review of algorithm auditing studies and identification
|
||
of trends in their methodological approaches.</em></li>
|
||
<li><a href="https://arxiv.org/pdf/2402.12572v1.pdf">FairProof:
|
||
Confidential and Certifiable Fairness for Neural Networks</a> - (Arxiv)
|
||
<em>Proposes an alternative paradigm to traditional auditing using
|
||
crytographic tools like Zero-Knowledge Proofs; gives a system called
|
||
FairProof for verifying fairness of small neural networks.</em></li>
|
||
<li><a
|
||
href="https://grodino.github.io/projects/manipulated-audits/preprint.pdf">Under
|
||
manipulations, are some AI models harder to audit?</a> - (SATML)
|
||
<em>Relates the difficulty of black-box audits to the capacity of the
|
||
targeted models, using the Rademacher complexity.</em></li>
|
||
<li><a href="https://arxiv.org/pdf/2310.07219.pdf">Improved Membership
|
||
Inference Attacks Against Language Classification Models</a> - (ICLR)
|
||
<em>Presents a framework for running membership inference attacks
|
||
against classifier, in audit mode.</em></li>
|
||
<li><a href="https://arxiv.org/pdf/2305.17570.pdf">Auditing Fairness by
|
||
Betting</a> - (Neurips) <a
|
||
href="https://github.com/bchugg/auditing-fairness">[Code]</a>
|
||
<em>Sequential methods that allows for the continuous monitoring of
|
||
incoming data from a black-box classifier or regressor.</em> ###
|
||
2023</li>
|
||
<li><a href="https://neurips.cc/virtual/2023/poster/70925">Privacy
|
||
Auditing with One (1) Training Run</a> - (NeurIPS - best paper) <em>A
|
||
scheme for auditing differentially private machine learning systems with
|
||
a single training run.</em></li>
|
||
<li><a
|
||
href="https://www.sciencedirect.com/science/article/pii/S0306457322003259">Auditing
|
||
fairness under unawareness through counterfactual reasoning</a> -
|
||
(Information Processing & Management) <em>Shows how to unveil
|
||
whether a black-box model, complying with the regulations, is still
|
||
biased or not.</em></li>
|
||
<li><a href="https://arxiv.org/pdf/2206.04740.pdf">XAudit : A
|
||
Theoretical Look at Auditing with Explanations</a> - (Arxiv)
|
||
<em>Formalizes the role of explanations in auditing and investigates if
|
||
and how model explanations can help audits.</em></li>
|
||
<li><a href="https://arxiv.org/pdf/2305.12620.pdf">Keeping Up with the
|
||
Language Models: Robustness-Bias Interplay in NLI Data and Models</a> -
|
||
(Arxiv) <em>Proposes a way to extend the shelf-life of auditing datasets
|
||
by using language models themselves; also finds problems with the
|
||
current bias auditing metrics and proposes alternatives – these
|
||
alternatives highlight that model brittleness superficially increased
|
||
the previous bias scores.</em></li>
|
||
<li><a href="https://dl.acm.org/doi/pdf/10.1145/3580305.3599454">Online
|
||
Fairness Auditing through Iterative Refinement</a> - (KDD) <em>Provides
|
||
an adaptive process that automates the inference of probabilistic
|
||
guarantees associated with estimating fairness metrics.</em></li>
|
||
<li><a
|
||
href="https://people.cs.umass.edu/~amir/papers/CCS23-LM-stealing.pdf">Stealing
|
||
the Decoding Algorithms of Language Models</a> - (CCS) <em>Steal the
|
||
type and hyperparameters of the decoding algorithms of a LLM.</em></li>
|
||
<li><a
|
||
href="https://link.springer.com/epdf/10.1007/s13278-023-01105-9?sharing_token=h-O-asHI49VUWS9FxN1Gsve4RwlQNchNByi7wbcMAY6I98PKW1PqhFQJ_JqQyk3TrB05qDb3LUzMDmKOgrupccQliViDle-rwKEi2MZ8xBViaAQhyN41oZBKLLeXchoeIW2kklVHC094I5KD8pxja4-if6-iB0uAI1FnqnYoxjU%3D">Modeling
|
||
rabbit‑holes on YouTube</a> - (SNAM) <em>Models the trapping dynamics of
|
||
users in rabbit holes in YouTube, and provides a measure of this
|
||
enclosure.</em></li>
|
||
<li><a href="https://dl.acm.org/doi/full/10.1145/3568392">Auditing
|
||
YouTube’s Recommendation Algorithm for Misinformation Filter Bubbles</a>
|
||
- (Transactions on Recommender Systems) <em>What it takes to “burst the
|
||
bubble,” i.e., revert the bubble enclosure from
|
||
recommendations.</em></li>
|
||
<li><a href="https://arxiv.org/pdf/2308.02129.pdf">Auditing Yelp’s
|
||
Business Ranking and Review Recommendation Through the Lens of
|
||
Fairness</a> - (Arxiv) <em>Audits the fairness of Yelp’s business
|
||
ranking and review recommendation systems, with demographic parity,
|
||
exposure, and statistical tests such as quantile linear and logistic
|
||
regression.</em></li>
|
||
<li><a
|
||
href="https://openreview.net/pdf?id=iIfDQVyuFD">Confidential-PROFITT:
|
||
Confidential PROof of FaIr Training of Trees</a> - (ICLR) <em>Proposes
|
||
fair decision tree learning algorithms along with zero-knowledge proof
|
||
protocols to obtain a proof of fairness on the audited server.</em></li>
|
||
<li><a href="https://arxiv.org/pdf/2302.03251.pdf">SCALE-UP: An
|
||
Efficient Black-box Input-level Backdoor Detection via Analyzing Scaled
|
||
Prediction Consistency</a> - (ICLR) <em>Considers backdoor detection
|
||
under the black-box setting in machine learning as a service (MLaaS)
|
||
applications.</em> ### 2022</li>
|
||
<li><a
|
||
href="https://ojs.aaai.org/index.php/ICWSM/article/view/19300/19072">Two-Face:
|
||
Adversarial Audit of Commercial Face Recognition Systems</a> - (ICWSM)
|
||
<em>Performs an adversarial audit on multiple systems APIs and datasets,
|
||
making a number of concerning observations.</em></li>
|
||
<li><a
|
||
href="https://journals.sagepub.com/doi/10.1177/01655515221093029">Scaling
|
||
up search engine audits: Practical insights for algorithm auditing</a> -
|
||
(Journal of Information Science) <a
|
||
href="https://github.com/gesiscss/WebBot">(Code)</a> <em>Audits multiple
|
||
search engines using simulated browsing behavior with virtual
|
||
agents.</em></li>
|
||
<li><a href="https://openreview.net/pdf?id=OUz_9TiTv9j">A zest of lime:
|
||
towards architecture-independent model distances</a> - (ICLR)
|
||
<em>Measures the distance between two remote models using
|
||
LIME.</em></li>
|
||
<li><a
|
||
href="https://proceedings.mlr.press/v162/yan22c/yan22c.pdf">Active
|
||
Fairness Auditing</a> - (ICML) <em>Studies of query-based auditing
|
||
algorithms that can estimate the demographic parity of ML models in a
|
||
query-efficient manner.</em></li>
|
||
<li><a
|
||
href="https://proceedings.neurips.cc/paper/2021/file/da94cbeff56cfda50785df477941308b-Paper.pdf">Look
|
||
at the Variance! Efficient Black-box Explanations with Sobol-based
|
||
Sensitivity Analysis</a> - (NeurIPS) <em>Sobol indices provide an
|
||
efficient way to capture higher-order interactions between image regions
|
||
and their contributions to a (black box) neural network’s prediction
|
||
through the lens of variance.</em></li>
|
||
<li><a href="https://arxiv.org/pdf/2204.10920.pdf">Your Echos are Heard:
|
||
Tracking, Profiling, and Ad Targeting in the Amazon Smart Speaker
|
||
Ecosystem</a> - (arxiv) <em>Infers a link between the Amazon Echo system
|
||
and the ad targeting algorithm.</em> ### 2021</li>
|
||
<li><a href="https://arxiv.org/pdf/2102.00141.pdf">When the Umpire is
|
||
also a Player: Bias in Private Label Product Recommendations on
|
||
E-commerce Marketplaces</a> - (FAccT) <em>Do Amazon private label
|
||
products get an unfair share of recommendations and are therefore
|
||
advantaged compared to 3rd party products?</em></li>
|
||
<li><a href="https://arxiv.org/pdf/2105.02980.pdf">Everyday Algorithm
|
||
Auditing: Understanding the Power of Everyday Users in Surfacing Harmful
|
||
Algorithmic Behaviors</a> - (CHI) <em>Makes the case for “everyday
|
||
algorithmic auditing” by users.</em></li>
|
||
<li><a
|
||
href="https://www.cs.bu.edu/faculty/crovella/paper-archive/minimization-audit-Neurips21.pdf">Auditing
|
||
Black-Box Prediction Models for Data Minimization Compliance</a> -
|
||
(NeurIPS) <em>Measures the level of data minimization satisfied by the
|
||
prediction model using a limited number of queries.</em></li>
|
||
<li><a href="https://arxiv.org/abs/2012.05101">Setting the Record
|
||
Straighter on Shadow Banning</a> - (INFOCOM) <a
|
||
href="https://gitlab.enseeiht.fr/bmorgan/infocom-2021">(Code)</a>
|
||
<em>Considers the possibility of shadow banning in Twitter (ie, the
|
||
moderation black-box algorithm), and measures the probability of several
|
||
hypothesis.</em></li>
|
||
<li><a href="https://arxiv.org/pdf/2012.07805.pdf">Extracting Training
|
||
Data from Large Language Models</a> - (USENIX Security) <em>Extract
|
||
verbatim text sequences from the GPT-2 model’s training data.</em></li>
|
||
<li><a
|
||
href="https://www.sciencedirect.com/science/article/pii/S030645732100145X?casa_token=oyjFKij269MAAAAA:w_ohScpMPNMnkDdzBqAIod5QfBgQlq5Ht9mMRSOydZpOgNG-i1yuqEmBjWN__38gOGmjNL7dVT0">FairLens:
|
||
Auditing black-box clinical decision support systems</a> - (Information
|
||
Processing & Management) <em>Presents a pipeline to detect and
|
||
explain potential fairness issues in Clinical DSS, by comparing
|
||
different multi-label classification disparity measures.</em></li>
|
||
<li><a
|
||
href="https://dl.acm.org/doi/abs/10.1145/3447535.3462491">Auditing
|
||
Algorithmic Bias on Twitter</a> - (WebSci).</li>
|
||
<li><a
|
||
href="https://proceedings.mlr.press/v139/neiswanger21a.html">Bayesian
|
||
Algorithm Execution: Estimating Computable Properties of Black-box
|
||
Functions Using Mutual Information</a> - (ICML) <em>A budget constrained
|
||
and Bayesian optimization procedure to extract properties out of a
|
||
black-box algorithm.</em> ### 2020</li>
|
||
<li><a
|
||
href="https://proceedings.neurips.cc/paper/2020/file/e8d66338fab3727e34a9179ed8804f64-Paper.pdf">Black-Box
|
||
Ripper: Copying black-box models using generative evolutionary
|
||
algorithms</a> - (NeurIPS) <em>Replicates the functionality of a
|
||
black-box neural model, yet with no limit on the amount of queries (via
|
||
a teacher/student scheme and an evolutionary search).</em></li>
|
||
<li><a
|
||
href="https://dl.acm.org/doi/pdf/10.1145/3351095.3372879">Auditing
|
||
radicalization pathways on</a> - (FAT<em>) </em>Studies the reachability
|
||
of radical channels from each others, using random walks on static
|
||
channel recommendations.*</li>
|
||
<li><a href="https://arxiv.org/abs/1912.07721">Adversarial Model
|
||
Extraction on Graph Neural Networks</a> - (AAAI Workshop on Deep
|
||
Learning on Graphs: Methodologies and Applications) <em>Introduces GNN
|
||
model extraction and presents a preliminary approach for this.</em></li>
|
||
<li><a href="https://rdcu.be/b6qB4">Remote Explainability faces the
|
||
bouncer problem</a> - (Nature Machine Intelligence volume 2,
|
||
pages529–539) <a
|
||
href="https://github.com/erwanlemerrer/bouncer_problem">(Code)</a>
|
||
<em>Shows the impossibility (with one request) or the difficulty to spot
|
||
lies on the explanations of a remote AI decision.</em></li>
|
||
<li><a
|
||
href="https://openaccess.thecvf.com/content_CVPR_2020/papers/Rahmati_GeoDA_A_Geometric_Framework_for_Black-Box_Adversarial_Attacks_CVPR_2020_paper.pdf">GeoDA:
|
||
a geometric framework for black-box adversarial attacks</a> - (CVPR) <a
|
||
href="https://github.com/thisisalirah/GeoDA">(Code)</a> <em>Crafts
|
||
adversarial examples to fool models, in a pure blackbox setup (no
|
||
gradients, inferred class only).</em></li>
|
||
<li><a
|
||
href="https://github.com/erwanlemerrer/erwanlemerrer.github.io/raw/master/files/imitation_blackbox_recommenders_netys-2020.pdf">The
|
||
Imitation Game: Algorithm Selectionby Exploiting Black-Box
|
||
Recommender</a> - (Netys) <a
|
||
href="https://github.com/gdamaskinos/RecRank">(Code)</a> <em>Parametrize
|
||
a local recommendation algorithm by imitating the decision of a remote
|
||
and better trained one.</em></li>
|
||
<li><a
|
||
href="https://ojs.aaai.org/index.php/ICWSM/article/view/7277">Auditing
|
||
News Curation Systems:A Case Study Examining Algorithmic and Editorial
|
||
Logic in Apple News</a> - (ICWSM) <em>Audit study of Apple News as a
|
||
sociotechnical news curation system (trending stories
|
||
section).</em></li>
|
||
<li><a
|
||
href="https://dl.acm.org/doi/pdf/10.1145/3375627.3375852">Auditing
|
||
Algorithms: On Lessons Learned and the Risks of DataMinimization</a> -
|
||
(AIES) <em>A practical audit for a well-being recommendation app
|
||
developed by Telefónica (mostly on bias).</em></li>
|
||
<li><a href="https://arxiv.org/pdf/2012.07805">Extracting Training Data
|
||
from Large Language Models</a> - (arxiv) <em>Performs a training data
|
||
extraction attack to recover individual training examples by querying
|
||
the language model.</em> ### 2019</li>
|
||
<li><a href="https://arxiv.org/abs/1711.01894">Adversarial Frontier
|
||
Stitching for Remote Neural Network Watermarking</a> - (Neural Computing
|
||
and Applications) <a
|
||
href="https://github.com/dunky11/adversarial-frontier-stitching">(Alternative
|
||
implementation)</a> <em>Check if a remote machine learning model is a
|
||
“leaked” one: through standard API requests to a remote model, extract
|
||
(or not) a zero-bit watermark, that was inserted to watermark valuable
|
||
models (eg, large deep neural networks).</em></li>
|
||
<li><a href="https://arxiv.org/abs/1812.02766.pdf">Knockoff Nets:
|
||
Stealing Functionality of Black-Box Models</a> - (CVPR) <em>Ask to what
|
||
extent can an adversary steal functionality of such “victim” models
|
||
based solely on blackbox interactions: image in, predictions
|
||
out.</em></li>
|
||
<li><a href="https://par.nsf.gov/servlets/purl/10101277">Opening Up the
|
||
Black Box:Auditing Google’s Top Stories Algorithm</a> - (Flairs-32)
|
||
<em>Audit of the Google’s Top stories panel that pro-vides insights into
|
||
its algorithmic choices for selectingand ranking news
|
||
publisher</em></li>
|
||
<li><a href="https://arxiv.org/pdf/1906.03397.pdf">Making targeted
|
||
black-box evasion attacks effective andefficient</a> - (arXiv)
|
||
<em>Investigates how an adversary can optimally use its query budget for
|
||
targeted evasion attacks against deep neural networks.</em></li>
|
||
<li><a
|
||
href="https://research.fb.com/wp-content/uploads/2019/05/Online-Learning-for-Measuring-Incentive-Compatibility-in-Ad-Auctions.pdf">Online
|
||
Learning for Measuring Incentive Compatibility in Ad Auctions</a> -
|
||
(WWW) <em>Measures the incentive compatible- (IC) mechanisms (regret) of
|
||
black-box auction platforms.</em></li>
|
||
<li><a href="https://arxiv.org/abs/1903.00317">TamperNN: Efficient
|
||
Tampering Detection of Deployed Neural Nets</a> - (ISSRE) <em>Algorithms
|
||
to craft inputs that can detect the tampering with a remotely executed
|
||
classifier model.</em></li>
|
||
<li><a href="https://arxiv.org/pdf/1903.03916.pdf">Neural Network Model
|
||
Extraction Attacks in Edge Devicesby Hearing Architectural Hints</a> -
|
||
(arxiv) <em>Through the acquisition of memory access events from bus
|
||
snooping, layer sequence identification bythe LSTM-CTC model, layer
|
||
topology connection according to the memory access pattern, and layer
|
||
dimension estimation under data volume constraints, it demonstrates one
|
||
can accurately recover the a similar network architecture as the attack
|
||
starting point</em></li>
|
||
<li><a
|
||
href="https://ieeexplore.ieee.org/abstract/document/8851798">Stealing
|
||
Knowledge from Protected Deep Neural Networks Using Composite Unlabeled
|
||
Data</a> - (ICNN) <em>Composite method which can be used to attack and
|
||
extract the knowledge ofa black box model even if it completely conceals
|
||
its softmaxoutput.</em></li>
|
||
<li><a href="https://dl.acm.org/citation.cfm?id=3354261">Neural Network
|
||
Inversion in Adversarial Setting via Background Knowledge Alignment</a>
|
||
- (CCS) <em>Model inversion approach in the adversary setting based on
|
||
training an inversion model that acts as aninverse of the original
|
||
model. With no fullknowledge about the original training data, an
|
||
accurate inversion is still possible by training the inversion model on
|
||
auxiliary samplesdrawn from a more generic data distribution.</em> ###
|
||
2018</li>
|
||
<li><a href="https://arxiv.org/abs/1711.00399">Counterfactual
|
||
Explanations without Opening the Black Box: Automated Decisions and the
|
||
GDPR</a> - (Harvard Journal of Law & Technology) <em>To explain a
|
||
decision on x, find a conterfactual: the closest point to x that changes
|
||
the decision.</em></li>
|
||
<li><a href="https://arxiv.org/abs/1710.06169">Distill-and-Compare:
|
||
Auditing Black-Box Models Using Transparent Model Distillation</a> -
|
||
(AIES) <em>Treats black box models as teachers, training transparent
|
||
student models to mimic the risk scores assigned by black-box
|
||
models.</em></li>
|
||
<li><a href="https://arxiv.org/abs/1711.01768">Towards
|
||
Reverse-Engineering Black-Box Neural Networks</a> - (ICLR) <a
|
||
href="https://github.com/coallaoh/WhitenBlackBox">(Code)</a> <em>Infer
|
||
inner hyperparameters (eg number of layers, non-linear activation type)
|
||
of a remote neural network model by analysing its response patterns to
|
||
certain inputs.</em></li>
|
||
<li><a
|
||
href="https://www.sciencedirect.com/science/article/pii/S092523121830136X">Data
|
||
driven exploratory attacks on black box classifiers in adversarial
|
||
domains</a> - (Neurocomputing) <em>Reverse engineers remote classifier
|
||
models (e.g., for evading a CAPTCHA test).</em></li>
|
||
<li><a href="https://arxiv.org/pdf/1806.08867.pdf">xGEMs: Generating
|
||
Examplars to Explain Black-Box Models</a> - (arXiv) <em>Searches bias in
|
||
the black box model by training an unsupervised implicit generative
|
||
model. Thensummarizes the black-box model behavior quantitatively by
|
||
perturbing data samples along the data manifold.</em></li>
|
||
<li><a href="https://arxiv.org/pdf/1801.07386">Learning Networks from
|
||
Random Walk-Based Node Similarities</a> - (NIPS) <em>Reversing graphs by
|
||
observing some random walk commute times.</em></li>
|
||
<li><a
|
||
href="https://rd.springer.com/chapter/10.1007/978-3-030-00374-6_6">Identifying
|
||
the Machine Learning Family from Black-Box Models</a> - (CAEPIA)
|
||
<em>Determines which kind of machine learning model is behind the
|
||
returned predictions.</em></li>
|
||
<li><a href="https://arxiv.org/pdf/1812.11720.pdf">Stealing Neural
|
||
Networks via Timing Side Channels</a> - (arXiv)
|
||
<em>Stealing/approximating a model through timing attacks usin
|
||
queries.</em></li>
|
||
<li><a href="https://arxiv.org/abs/1806.05476">Copycat CNN: Stealing
|
||
Knowledge by Persuading Confession with Random Non-Labeled Data</a> -
|
||
(IJCNN) <a href="https://github.com/jeiks/Stealing_DL_Models">(Code)</a>
|
||
<em>Stealing black-box models (CNNs) knowledge by querying them with
|
||
random natural images (ImageNet and Microsoft-COCO).</em></li>
|
||
<li><a href="https://dl.acm.org/doi/10.1145/3178876.3186143">Auditing
|
||
the Personalization and Composition of Politically-Related Search Engine
|
||
Results Pages</a> - (WWW) <em>A Chrome extension to survey participants
|
||
and collect the Search Engine Results Pages (SERPs) and autocomplete
|
||
suggestions, for studying personalization and composition.</em> ###
|
||
2017</li>
|
||
<li><a href="https://dl.acm.org/authorize.cfm?key=N21772">Uncovering
|
||
Influence Cookbooks : Reverse Engineering the Topological Impact in Peer
|
||
Ranking Services</a> - (CSCW) <em>Aims at identifying which centrality
|
||
metrics are in use in a peer ranking service.</em></li>
|
||
<li><a href="https://arxiv.org/abs/1704.08991">The topological face of
|
||
recommendation: models and application to bias detection</a> - (Complex
|
||
Networks) <em>Proposes a bias detection framework for items recommended
|
||
to users.</em></li>
|
||
<li><a href="http://ieeexplore.ieee.org/document/7958568/">Membership
|
||
Inference Attacks Against Machine Learning Models</a> - (Symposium on
|
||
Security and Privacy) <em>Given a machine learning model and a record,
|
||
determine whether this record was used as part of the model’s training
|
||
dataset or not.</em></li>
|
||
<li><a href="https://dl.acm.org/citation.cfm?id=3053009">Practical
|
||
Black-Box Attacks against Machine Learning</a> - (Asia CCS)
|
||
<em>Understand how vulnerable is a remote service to adversarial
|
||
classification attacks.</em> ### 2016</li>
|
||
<li><a
|
||
href="https://www.andrew.cmu.edu/user/danupam/datta-sen-zick-oakland16.pdf">Algorithmic
|
||
Transparency via Quantitative Input Influence: Theory and Experiments
|
||
with Learning Systems</a> - (IEEE S&P) <em>Evaluate the individual,
|
||
joint and marginal influence of features on a model using shapley
|
||
values.</em></li>
|
||
<li><a href="https://arxiv.org/abs/1602.07043">Auditing Black-Box Models
|
||
for Indirect Influence</a> - (ICDM) <em>Evaluate the influence of a
|
||
variable on a black-box model by “cleverly” removing it from the dataset
|
||
and looking at the accuracy gap</em></li>
|
||
<li><a href="https://arxiv.org/abs/1611.04967">Iterative Orthogonal
|
||
Feature Projection for Diagnosing Bias in Black-Box Models</a> - (FATML
|
||
Workshop) <em>Performs feature ranking to analyse black-box
|
||
models</em></li>
|
||
<li><a href="http://datworkshop.org/papers/dat16-final22.pdf">Bias in
|
||
Online Freelance Marketplaces: Evidence from TaskRabbit</a> - (dat
|
||
workshop) <em>Measures the TaskRabbit’s search algorithm rank.</em></li>
|
||
<li><a
|
||
href="https://www.usenix.org/conference/usenixsecurity16/technical-sessions/presentation/tramer">Stealing
|
||
Machine Learning Models via Prediction APIs</a> - (Usenix Security) <a
|
||
href="https://github.com/ftramer/Steal-ML">(Code)</a> <em>Aims at
|
||
extracting machine learning models in use by remote services.</em></li>
|
||
<li><a href="https://arxiv.org/pdf/1602.04938v3.pdf">“Why Should I Trust
|
||
You?”Explaining the Predictions of Any Classifier</a> - (arXiv) <a
|
||
href="https://github.com/marcotcr/lime-experiments">(Code)</a>
|
||
<em>Explains a blackbox classifier model by sampling around data
|
||
instances.</em></li>
|
||
<li><a href="http://ieeexplore.ieee.org/document/7546497/">Back in
|
||
Black: Towards Formal, Black Box Analysis of Sanitizers and Filters</a>
|
||
- (Security and Privacy) <em>Black-box analysis of sanitizers and
|
||
filters.</em></li>
|
||
<li><a href="http://ieeexplore.ieee.org/document/7546525/">Algorithmic
|
||
Transparency via Quantitative Input Influence: Theory and Experiments
|
||
with Learning Systems</a> - (Security and Privacy) <em>Introduces
|
||
measures that capture the degree of influence of inputs on outputs of
|
||
the observed system.</em></li>
|
||
<li><a href="https://mislove.org/publications/Amazon-WWW.pdf">An
|
||
Empirical Analysis of Algorithmic Pricing on Amazon Marketplace</a> -
|
||
(WWW) <a href="http://personalization.ccs.neu.edu">(Code)</a>
|
||
<em>Develops a methodology for detecting algorithmic pricing, and use it
|
||
empirically to analyze their prevalence and behavior on Amazon
|
||
Marketplace.</em> ### 2015</li>
|
||
<li><a href="https://arxiv.org/abs/1412.3756">Certifying and Removing
|
||
Disparate Impact</a> - (SIGKDD) <em>Proposes SVM-based methods to
|
||
certify absence of bias and methods to remove biases from a
|
||
dataset.</em></li>
|
||
<li><a href="https://dl.acm.org/citation.cfm?id=2815681">Peeking Beneath
|
||
the Hood of Uber</a> - (IMC) <em>Infer implementation details of Uber’s
|
||
surge price algorithm.</em> ### 2014</li>
|
||
<li><a href="">A peek into the black box: exploring classifiers by
|
||
randomization</a> - (Data Mining and Knowledge Discovery journal) (<a
|
||
href="https://github.com/tsabsch/goldeneye">code</a>) <em>Finds groups
|
||
of features that can be permuted without changing the output label of
|
||
predicted samples</em></li>
|
||
<li><a href="https://www.usenix.org/node/184394">XRay: Enhancing the
|
||
Web’s Transparency with Differential Correlation</a> - (USENIX Security)
|
||
<em>Audits which user profile data were used for targeting a particular
|
||
ad, recommendation, or price.</em> ### 2013</li>
|
||
<li><a href="https://dl.acm.org/citation.cfm?id=2488435">Measuring
|
||
Personalization of Web Search</a> - (WWW) <em>Develops a methodology for
|
||
measuring personalization in Web search result.</em></li>
|
||
<li><a
|
||
href="https://www.cs.bgu.ac.il/~sabatos/papers/SabatoSarwate13.pdf">Auditing:
|
||
Active Learning with Outcome-Dependent Query Costs</a> - (NIPS)
|
||
<em>Learns from a binary classifier paying only for negative
|
||
labels.</em></li>
|
||
</ul>
|
||
<h3 id="section-2">2012</h3>
|
||
<ul>
|
||
<li><a href="http://www.jmlr.org/papers/v13/nelson12a.html">Query
|
||
Strategies for Evading Convex-Inducing Classifiers</a> - (JMLR)
|
||
<em>Evasion methods for convex classifiers. Considers evasion
|
||
complexity.</em> ### 2008</li>
|
||
<li><a href="https://dl.acm.org/citation.cfm?id=1455806">Privacy Oracle:
|
||
a System for Finding Application Leakswith Black Box Differential
|
||
Testing</a> - (CCS) <em>Privacy Oracle: a system that uncovers
|
||
applications’ leaks of personal information in transmissions to
|
||
remoteservers.</em> ### 2005</li>
|
||
<li><a href="https://dl.acm.org/citation.cfm?id=1081950">Adversarial
|
||
Learning</a> - (KDD) <em>Reverse engineering of remote linear
|
||
classifiers, using membership queries.</em></li>
|
||
</ul>
|
||
<h2 id="related-events">Related Events</h2>
|
||
<h3 id="section-3">2025</h3>
|
||
<ul>
|
||
<li><a href="https://project.inria.fr/aimlai/">AIMLAI at ECML/PKDD
|
||
2025</a></li>
|
||
<li><a
|
||
href="https://aaai.org/conference/aaai/aaai-25/workshop-list/#ws06">AAAI
|
||
workshop on AI Governance: Alignment, Morality, and Law</a></li>
|
||
</ul>
|
||
<h3 id="section-4">2024</h3>
|
||
<ul>
|
||
<li><a href="https://www.ircg.msm.uni-due.de/ai/">1st International
|
||
Conference on Auditing and Artificial Intelligence</a></li>
|
||
<li><a href="https://regulatableml.github.io/">Regulatable ML Workshop
|
||
(RegML’24)</a></li>
|
||
</ul>
|
||
<h3 id="section-5">2023</h3>
|
||
<ul>
|
||
<li><a href="https://cscw-user-ai-auditing.github.io/">Supporting User
|
||
Engagement in Testing, Auditing, and Contesting AI (CSCW User AI
|
||
Auditing)</a></li>
|
||
<li><a href="https://algorithmic-audits.github.io">Workshop on
|
||
Algorithmic Audits of Algorithms (WAAA)</a></li>
|
||
<li><a href="https://regulatableml.github.io/">Regulatable ML Workshop
|
||
(RegML’23)</a></li>
|
||
</ul>
|
||
<p><a
|
||
href="https://github.com/erwanlemerrer/awesome-audit-algorithms">auditalgorithms.md
|
||
Github</a></p>
|