Connectionists: Global Knowledge Tracing Challenge @AAAI 2023

[Zitao Liu]

** Call for Participation ** - Global Knowledge Tracing Challenge @AAAI 2023

http://ai4ed.cc/competitions/aaai2023competition

---------------------
Background
---------------------
In this competition, we would like to call for researchers and
practitioners worldwide to investigate the opportunities of improving the
student assessment performance via knowledge tracing approaches with rich
side information.

Knowledge tracing (KT) is the task of using students’ historical learning
interaction data to model their knowledge mastery over time so as to make
predictions on their future interaction performance, shown in the following
Figure. Such predictive capabilities can potentially help students learn
better and faster when paired with high-quality learning materials and
instructions, which is crucial for building next-generation smart and
personalized education.

However, even though a large body of deep learning based KT models are
proposed, the majority of existing baselines don't utilize the rich
auxiliary side information in educational contexts. Various auxiliary side
information could be extracted as external knowledge and integrated with
the DLKT models. These auxiliary knowledge are expected to improve DLKT
performance, which can be considered as follows:


   - Question side information: (1) question text content; (2) latent
   question variations with respect to each KC; (3) question difficulty level;
   and (4) relations among questions.
   - Student side information: (1) historical successful and failed
   attempts; (2) recent attempts; (3) students’ learning ability; and (4)
   individualized prior knowledge of students.
   - KC side information: (1) latent knowledge representation; and (2)
   relations among KCs.


---------------------
Data & Task Description
---------------------
We released a large student assessment dataset with rich textual and
structural information (well anonymized). The data is collected from
students in grade 3 math classes at Xueersi from TAL Education Group that
is a leading smart learning solutions provider in China with global
footprints. Specifically, *the dataset is made up of 18,066 students, 7,652
questions, 1,175 KCs and 5,549,635 interactions/responses*. The average
student historical sequence length is 307.19. 79.47% responses are with
positive labels, i.e., students correctly answer the questions.

Each question is associated with question text content, the explanations of
question answers and its corresponding KCs. The relevant KCs are
hierarchically structured. All the textual characters are anonymized and
mapped into internal unique integer tokens.

---------------------
Evaluation Task
---------------------
There are 3,613 student interaction sequences in the test set. For each
interaction sequence, the first 50% of interactions are known and the task
is to predict the rest 50% interactions’ responses, i.e., 0 or 1.

We will choose to use AUC score as the main evaluation metric for this
competition.

---------------------
Baseline Results
---------------------
We have used pyKT to run DKT and AKT on the aforementioned training data
and evaluate on our public test sets. All these approaches are purely
trained with question/KC ids and student responses without any auxiliary
information, such as question content. The results are used as baseline
results for this competition:

Model   Non-Accumulative    Accumulative
DKT                0.6801                 0.7086
AKT                0.7812                 0.7692
Majority           0.7381                      -

We provide easy to use codes and detailed instructions in:
https://github.com/pykt-team/pykt-toolkit/tree/main/examples/competitions/aaai2023_competition

---------------------
Competition Schedule
---------------------

   - November 17, 2022 - Start Date.
   - December 31, 2022 - Final submission deadline.
   - January 2, 2023 - Final competition results announced.


All deadlines are at 11:59 PM UTC on the corresponding day unless otherwise
noted.The competition organizers reserve the right to update the contest
timeline if they deem it necessary.

---------------------
Award
---------------------

   - We will provide cash prizes for the top-3 teams (1st place: $1000; 2nd
   place: $600 ; 3rd place $300)
   - An official certificate will be awarded to the top-3 teams.
   - The top-3 teams will be invited to give oral presentations during AAAI
   2023.
   - The first author of the top-3 teams will be invited to contribute to a
   competition review paper.


Note: The top-3 teams should make their training and testing code publicly
available for verification after the testing submission deadline.

---------------------
Organizers
----------------

Qiongqiong Liu TAL Education Group, China
Jiahao Chen TAL Education Group, China
Shuyan Huang TAL Education Group, China
Zitao Liu TAL Education Group, China

Contact: pykt.team at gmail.com


Best Regards,

The pyKT Team

Dr. Zitao Liu | http://www.zitaoliu.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20221102/19a0b6c6/attachment.html>