I’m a Research Engineer at Meta (2022-present) working on Large Language Models. Previous work includes LLama 2, 3, 4 and Galactica
Experience
- I’m currently a Research Engineer at Meta working mainly in post training. Areas include: - Factuality: agentic retrieval over web search for factuality. - Hallucination reduction: keeping LLMs faithful to their training data. - Reasoning: training LLMs to reason with long chain of thoughts using sparse rewards.
- I’ve advised startups on Machine Learning:
- I was the Machine Learning Lead at Genie AI 2018-2022.
- Lead a 2 year InnovateUK collaboration with Marta Kwiatkowska’s team at Oxford in explainability and robustness in NLP. We’ve developed a novel way of explaining transformer models based on a game based framework, published a way of assessing robustness of machine learning models to adversarial examples using a monte carlo tree search, and developed techniques to explain machine learning models decisions causally.
- Lead a 2 year InnovateUK collaboration with the computational privacy group at Imperial college. We’ve developed techniques to assess vulnerabilities to authorship attribution attacks on text (automatically determining the author of a piece of text from their writing style), and potential ways to mitigate these risks.
- Developed countless proof of concepts of intelligent potential features and productionized the most promising into the Genie product.
- Worked with great advisors such as Jun Wang, Former Supreme Court President Lord Neurberger, and Adam Ziegler who created case.law
- Lead the technical due dillegence aspects of several funding rounds.
- Previously founder of legal tech company through EF (LD10). We managed to raise a bit of money and got some traction, but it was ultimately unsucessful and we pulled the plug. I learned a lot, and remain plugged into the startup ecosystem.
- Previously worked at Signal AI where I was researching reported speach detection in news (detecting quoations that are indirect and not within quotation marks).
- Cut my software teeth as a Java developer at IBlocks and as a software engineer at TNG Technology Consulting in Germany.
I’m an alrounder technologist with extensive experience in taking products from conception, product development and production. I’m a polyglot software engineer who can write well engineered software, perform devops and deploy to production.
Publications
- What Characterizes Effective Reasoning? Revisiting Length, Review, and Structure of CoT
- Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinations
- HalluLens: LLM Hallucination Benchmark
- The Llama 3 Herd of Models
- Llama 2: Open Foundation and Fine-Tuned Chat Models
- Improving clinical trial design using interpretable machine learning based prediction of early trial termination
- Galactica: A Large Language Model for Science
- Assessing Robustness of Text Classification through Maximal Safe Radius Computation