[SydPhil] MQ Philosophy WIP

[Emily Hughes]

Dear all,



You are warmly invited to the next MQ Philosophy Work in Progress (WIP)
seminar which will be given by *Raphaël Millière (University of Oxford).*



The details are as follows:

Date: Tuesday 18th of August

Time:   13:00-14:00

Room: 17WW 113



Zoom Link:
*https://url.au.m.mimecastprotect.com/s/y4RfCK1DvKTBR9LymSMfBoh5_gs9?domain=macquarie.zoom.us
<https://url.au.m.mimecastprotect.com/s/vP3iCL7EwMfmzXLD4SqhDWhyPQQV?domain=aus01.safelinks.protection.outlook.com>



*Title: *A Bayesian Framework for Comparative Cognitive Science (joint work
with Jennifer Hu)



*Abstract:* Suppose a language model and human subjects both achieve high
performance on a test designed to measure a given cognitive ability (e.g.,
analogical reasoning or theory of mind). Should we conclude that they both
have that ability to the same degree? Performance data alone can't settle
this question. Cognitive evaluation is an inverse problem: the target
ability is a latent variable that is not directly observed, and many hidden
causes can produce the same pattern of performance. Memorization, surface
cues, and response bias can enable good performance without the target
ability, while auxiliary task demands or measurement error can result in
observed failure despite the ability. We propose a Bayesian framework to
formalise the evaluation of cognitive capacities in humans, animals, and
machines. In this framework, a pattern of performance provides evidence
only through a comparison. The observed result supports attributing an
ability to the extent that it was more expected under a predictive model of
the subject with the ability than under a model of the subject without it.
This ratio (the Bayes factor) depends on modelling credible routes to
success and failure under each hypothesis, and comes apart from prior
beliefs about whether subject has the ability. It follows that the same
pattern of evidence can carry different evidence for different subjects.
This framework explains what makes a cognitive test diagnostic, and how to
choose which experiment to perform next as function of the uncertainty it
is expected to remove about the subject. In particular, it suggests that
the relative evidential weight of behavioural and mechanistic experiments
is quite different for human subjects and for artificial neural networks.
Beyond the framework's value as a regulative ideal for comparative
cognitive science, we review some of its practical implications for
experimental design.


Further information about the seminar series including details on future
talks can be found at the seminar webpage:
*https://url.au.m.mimecastprotect.com/s/-RP7CMwGxOtRn9jyWsJiZ9h8ggXy?domain=mqphilosophy.github.io
<https://url.au.m.mimecastprotect.com/s/5rRhCNLJyQUEg9oqXURs2OhysIzk?domain=mqphilosophy.github.io>

We look forward to seeing you there.

*Dr Emily Hughes *(she/her)

ARC Discovery Early Career Research Award (DECRA) Fellow in Philosophy

School of Humanities, Faculty of Arts

Michael Kirby Building, 17 Wally's Walk

Level 2, Room 233

Macquarie University – Wallumattagal Campus, Dharug Country

NSW 2109 Australia

E: emily.hughes at mq.edu.au

W: https://url.au.m.mimecastprotect.com/s/vdanCOMKzVT0Bw3L4uPtB2hG1ExE?domain=researchers.mq.edu.au



*I acknowledge that Macquarie University stands on the land of the Dharug
Nation, land that was never ceded. I pay my respects to the Dharug
people, the Wallumattagal clan, and their Elders past and present. Always
was, always will be, Aboriginal land.*