AI Projects

• LLM and RAG based context generation
• Using existing overview of past projects
• Using Ollama with Llama 3.2 and RAG
• Generation of context and recommendations for project and partners
• Internal tool for the support of new projects
• Enabling source identification, output context

• LLM based test case generation
• Increased efficiency of test case generation using LLM
• Evaluation of test case generation quality with quality criteria defined in ISO 26262 and ISO 29119, for Quality of Content (QoC)
• LLM uncertainty evaluation using Top-K-verbalize confidence and Vanilla prompting with perturbations Chain of thought prompting
• Evaluation on GSM8k, Business Ethics and professional law datasets
• Performance evaluation and comparison on available open source LLM models (Pixtral 12B, Llama, Gemma)
• Collaborator: Vidya Jois Manjunath

• Evaluation of out-of-scope uncertainty at runtime using Deep Knowledge and SafeML
• Runtime evaluation of operational context relevance to the design time context using SafeML
• Use cases: Emergency search and rescue using drones, and autonomous indoor transport robot
• Using Ultralytics YoloV8 for object detection, deployed on Nvidia Jetson
• Collaborators: Dr. Sondess Missaoui, Dr. Panagiota Nikolaou, PAL robotics

• Runtime evaluation of neuron-based uncertainties using Deep Knowledge
• Deep Knowledge: a test coverage criterion that enables effective quality assessments of DNNs.
• Focuses on test coverage to assess and mitigate risk before deployment
• Uses generalization theory principles, using a test adequacy criterion to enable fault detection and vulnerabilities detection
• Using Deep Knowledge at runtime for uncertainty monitoring
• Collaborators: Dr. Sondess Missaoui

• Evaluation of Adversarial robustness on SafeML
• Evaluation of effect of robustness on SafeML based out-of-context estimation
• Use case: time series-based sock price prediction
• Using adversarial attack such as Fast Gradient Sign Method, Basic Iterative Method etc
• SafeML-based robustness estimate StaDRo remained unaffected to all the selected adversarial attacks
• Collaborators: Sahil Ratra

• Evaluation and comparison of Explainable AI methods
• Explainable AI for the vision and time series application
• Proposed a novel method of explainable AI called SMILE
• Evaluation of SMILE on time series application and comparison against related methods, such as TELMI and TimeSHAP
• Evaluation of SMILE and BaySMILE on vision application and comparison with the related methods (LIME, SHAP, BayLIME)
• Collaborators: Dr koorosh Aslansefat, Vidushi jain, Ameer Khan

• Reliability and robustness estimate of time series forecasting
• Accounting Dataset shift in time series application using SafeML based statistical distance measure
• On Stock price prediction use case, developed using LSTM and GRU
• Relationship between Statistical Distance Dissimilarity (SDD) and performance
• Using SDD and performance relation to estimate the Statistical Distance Based Reliability (StaDRE) and Statistical Distance based Robustness (StaDRo)
• Collaborators: Akshatha Ambekar

• Using statistical distance to detect out of scope events
• Need of monitoring solutions beyond development, to account for runtime uncertainties
• Scope compliance uncertainties: uncertainties resulting from the application of the model beyond its scope
• SafeML: Using statistical distance measures to account scope compliance uncertainties
• Evaluation of SafeML on German traffic sign benchmark, and Person detection example from COCO dataset
• Using AlexNet and YOLO based detector (in tensorflow and pytorch)
• Collaborators: Dr. Ioannis Sorokos, Dr. Koorosh Aslansefat