[{"command":"insert","method":null,"selector":".course-modal-ajax-placeholder-959631","data":"\u003Cdiv class=\u0022container content-rte\u0022\u003E\n  \u003Cdiv class=\u0022row\u0022\u003E\n    \u003Cdiv class=\u0022box col-md-6 bg-primary text-white\u0022\u003E\n      \u003Ch2 class=\u0022h1 text-left\u0022\u003E  Grands mod\u00e8les de langage pour le code et la preuve\n\u003C\/h2\u003E\n                  \u003Cdiv class=\u0022field-spacing\u0022\u003E\n        \u003Cdiv class=\u0022field__label h3 dash\u0022\u003EECTS :\n          5\n        \u003C\/div\u003E\n      \u003C\/div\u003E\n                  \u003Cdiv class=\u0022field-spacing\u0022\u003E\n        \u003Cdiv class=\u0022field__label h3 dash\u0022\u003ESemester:\n          Semestre 2\n        \u003C\/div\u003E\n      \u003C\/div\u003E\n            \u003Cdiv class=\u0022field-spacing\u0022\u003E\n      \u003Cdiv class=\u0022field__label h3 dash\u0022\u003ED\u00e9tail du volume horaire\u003C\/div\u003E\n                        \u003Cdiv class=\u0022mt-5 text-justify\u0022\u003E\n                          \u003Cspan class=\u0022field__label h4 mt-5 \u0022\u003ELecture:\u003C\/span\u003E\n                                      \u003Cspan\u003E12\u003C\/span\u003E\n                      \u003C\/div\u003E\n                                              \u003Cdiv class=\u0022mt-5 text-justify\u0022\u003E\n                          \u003Cspan class=\u0022field__label h4 mt-5 \u0022\u003EDirected study:\u003C\/span\u003E\n                                      \u003Cspan\u003E12\u003C\/span\u003E\n                      \u003C\/div\u003E\n                                                            \u003C\/div\u003E\n        \u003Cdiv class=\u0022field-spacing\u0022\u003E\n      \u003Cdiv class=\u0022field__label h3 dash\u0022\u003ELangue d\u0026#039;enseignement\u003C\/div\u003E\n                        \u003Cdiv class=\u0022mt-5 text-justify\u0022\u003E\n                                      \u003Cspan\u003EAnglais\u003C\/span\u003E\n                      \u003C\/div\u003E\n                  \u003C\/div\u003E\n        \u003Cdiv class=\u0022field-spacing\u0022\u003E\n      \u003Cdiv class=\u0022field__label h3 dash\u0022\u003EEnseignement \u00e0 distance\u003C\/div\u003E\n                        \u003Cdiv class=\u0022mt-5 text-justify\u0022\u003E\n                                      \u003Cspan\u003Enon\u003C\/span\u003E\n                      \u003C\/div\u003E\n                  \u003C\/div\u003E\n      \u003C\/div\u003E\n    \u003Cdiv class=\u0022box col-md-6\u0022\u003E\n            \u003Cdiv class=\u0022field-spacing\u0022\u003E\n      \u003Cdiv class=\u0022field__label h3 dash\u0022\u003EPr\u00e9requis\u003C\/div\u003E\n                        \u003Cdiv class=\u0022mt-5 text-justify\u0022\u003E\n                                      \u003Cspan\u003E\u003Cp\u003ELe cours n\u00e9cessite de ma\u00eetriser de Python et Pytorch ainsi que des bases d\u2019alg\u00e8bre lin\u00e9aire et d\u2019optimisation.\u003C\/p\u003E\u003C\/span\u003E\n                      \u003C\/div\u003E\n                  \u003C\/div\u003E\n        \u003Cdiv class=\u0022field-spacing\u0022\u003E\n      \u003Cdiv class=\u0022field__label h3 dash\u0022\u003EProgramme \/ plan \/ contenus\u003C\/div\u003E\n                        \u003Cdiv class=\u0022mt-5 text-justify\u0022\u003E\n                                      \u003Cspan\u003E\u003Cul\u003E\n\u003Cli\u003EUnderstanding Large Language Models\n\u003Cul\u003E\n\u003Cli\u003EWhat are Large Language Models (LLMs)\u003C\/li\u003E\n\u003Cli\u003EOverview of applications: natural language processing tasks, code generation, proof generation\u003C\/li\u003E\n\u003Cli\u003EChallenges and limitations of LLMs\u003C\/li\u003E\n\u003C\/ul\u003E\n\u003C\/li\u003E\n\u003Cli\u003EWorking with Text Data\n\u003Cul\u003E\n\u003Cli\u003EData preprocessing techniques for text data\u003C\/li\u003E\n\u003Cli\u003ETokenization and vocabulary building\u003C\/li\u003E\n\u003C\/ul\u003E\n\u003C\/li\u003E\n\u003Cli\u003EAttention Mechanisms\n\u003Cul\u003E\n\u003Cli\u003ETypes of attention mechanisms (self-attention, multi-head attention)\u003C\/li\u003E\n\u003Cli\u003EEfficient implementations\u003C\/li\u003E\n\u003C\/ul\u003E\n\u003C\/li\u003E\n\u003Cli\u003EImplementing a GPT Model from scratch\n\u003Cul\u003E\n\u003Cli\u003EOverview of Generative Pre-trained Transformers (GPT)\u003C\/li\u003E\n\u003Cli\u003EArchitecture of GPT models (normalisation layers, residual connections, positional embedding)\u003C\/li\u003E\n\u003Cli\u003EPretraining on Unlabeled Data: scaling laws, evaluation metrics\u003C\/li\u003E\n\u003C\/ul\u003E\n\u003C\/li\u003E\n\u003Cli\u003EFine Tuning\n\u003Cul\u003E\n\u003Cli\u003EFine tuning for text classification\u003C\/li\u003E\n\u003Cli\u003EFine tuning with human feedback and reward models\u003C\/li\u003E\n\u003C\/ul\u003E\n\u003C\/li\u003E\n\u003Cli\u003ERetrieval Augmented Generation\n\u003Cul\u003E\n\u003Cli\u003EUsing retrieval mechanisms to enhance generation tasks\u003C\/li\u003E\n\u003Cli\u003ELanguage agents\u003C\/li\u003E\n\u003C\/ul\u003E\n\u003C\/li\u003E\n\u003Cli\u003ELLMs for code generation\n\u003Cul\u003E\n\u003Cli\u003ECode generation tasks (code completion, code summarization, code repair)\u003C\/li\u003E\n\u003Cli\u003ELearning from code-related feedback\u003C\/li\u003E\n\u003Cli\u003EInference algorithms for code\u003C\/li\u003E\n\u003C\/ul\u003E\n\u003C\/li\u003E\n\u003Cli\u003ELLMs and formal code\n\u003Cul\u003E\n\u003Cli\u003ELLMs for mathematics\u003C\/li\u003E\n\u003Cli\u003ENeural theorem proving\u003C\/li\u003E\n\u003Cli\u003EVerified code synthesis\u003C\/li\u003E\n\u003C\/ul\u003E\n\u003C\/li\u003E\n\u003C\/ul\u003E\u003C\/span\u003E\n                      \u003C\/div\u003E\n                  \u003C\/div\u003E\n        \u003Cdiv class=\u0022field-spacing\u0022\u003E\n      \u003Cdiv class=\u0022field__label h3 dash\u0022\u003EObjectifs d\u0026#039;apprentissage\u003C\/div\u003E\n                        \u003Cdiv class=\u0022mt-5 text-justify\u0022\u003E\n                                      \u003Cspan\u003E\u003Cp\u003EL\u2019objectif de ce cours est de pr\u00e9senter les Large Language Models, en insistant particuli\u00e8rement sur l\u2019architecture Transformers. La premi\u00e8re moiti\u00e9 du cours porte sur toutes les notions essentielles pour cette architecture, et l\u2019objectif est de coder en entier un Transformer utilisable, en ayant compris en profondeur et avec des outils math\u00e9matiques les m\u00e9canismes d\u2019attention. La seconde moiti\u00e9 du cours discute de certaines applications des LLMs dans des cadres formels (code, preuve).\u003C\/p\u003E\u003C\/span\u003E\n                      \u003C\/div\u003E\n                  \u003C\/div\u003E\n        \u003Cdiv class=\u0022field-spacing\u0022\u003E\n      \u003Cdiv class=\u0022field__label h3 dash\u0022\u003EOrganisation g\u00e9n\u00e9rale et modalit\u00e9s p\u00e9dagogiques\u003C\/div\u003E\n                        \u003Cdiv class=\u0022mt-5 text-justify\u0022\u003E\n                                      \u003Cspan\u003E\u003Cp\u003E8 s\u00e9ances\u003C\/p\u003E\u003C\/span\u003E\n                      \u003C\/div\u003E\n                  \u003C\/div\u003E\n                                \u003Cdiv class=\u0022field-spacing\u0022\u003E\n      \u003Cdiv class=\u0022field__label h3 dash\u0022\u003ENature de l\u0026#039;\u00e9valuation\u003C\/div\u003E\n                        \u003Cdiv class=\u0022mt-5 text-justify\u0022\u003E\n                                      \u003Cspan\u003EEvaluation Continue non Int\u00e9grale\u003C\/span\u003E\n                      \u003C\/div\u003E\n                  \u003C\/div\u003E\n      \u003C\/div\u003E\n  \u003C\/div\u003E\n\u003C\/div\u003E\n","settings":null}]