Presentation at ACM COMPUTE 2024 of my paper

Presented at ACM COMPUTE 2024, hosted at IIT Gandhinagar with the paper - HinglishEval: Evaluating the Effectiveness of Code-generation Models on Hinglish Prompts

The paper was co-authored with 3 of my batchmates under the supervision of Prof. Viraj Kumar . The research project (project repo) evaluated code generation LLMs on Hinglish prompts (a mix of English and Hindi, an Indian language). Hinglish is widely used in India. Hence, in order to explore the possible influence of code generation LLMs on computer science education in India, we benchmarked LLMs on the translated HumanEval dataset (which was semi-automatically translated to Hinglish) and utilized Item Response Theory to examine the results.

Click here for the presentation