Contents of this page:
- First study (1998)
- May 1999 study
- October 1999
- 2000 study
- 2001 study
- 2002 study
- 2003 study
- 2004 and 2005 studies
- 2006 study
- 2007 study
- to test CBM as a theory, and see whether it supports learning;
- to test the completeness and correctness of the constraint base;
- to evaluate the interface.
- to evaluate the feedback given to students;
- to test (again) how well CBM supports learning;
- to test the completeness and correctness of the constraint base.
- to evaluate the effectiveness of the pedagogical agent (SmartEgg);
- to evaluate the Bayesian student model and the problem selection strategy;
- to test (again) how well CBM supports learning.
Overview of SQL-Tutor
SQL-Tutor is a knowledge-based teaching system which supports students learning SQL. The intention was to provide an easy-to-use system that will adapt to the needs and learning abilities of individual students. The tailoring of instruction is done in two ways: by adapting the level of complexity of problems and by generating informative feedback messages. The architecture of the stand-alone version of the system is illustrated in the following figure.
SQL-Tutor is based on Constraint-Based Modeling (CBM), a student
modeling approach proposed by
In order to adapt to individual students, SQL-Tutor maintains a model for each student. A student model contains information about the history of previous sessions (like a list of problems solved correctly etc) and also contains a model of the student's knowledge, expressed in terms of constraints.
When a student logs in for the first time, the system creates a new model, and lets the student select a database and a problem to work on. Every student solutions is analyzed (ie, propagated to the constraint network), and the system reports on its findings. There are several levels of feedback messages:
Positive/negative
feedback simply informs the student whether his/her solution is correct or
not;
Error
flag informs the student about the clause of the SELECT statement an error
appeared in;
Hint
provides a general description of the error;
Detailed
hint, providing more information about the error;
Partial
solution gives the correct version of the clause where an error appeared;
Complete
solution present the ideal solution to the problem.
SQL-Tutor versions
The work on SQL-Tutor started in 1996, and now there are three version of the system available, developed in in Allegro Common Lisp, Franz Inc.
Solaris
version (interface developed in Tcl by Kendrick Elsey)
Windows
version (interface developed by Andrew King and Andre Renaud)
Web
Version (interface developed by Kurt Haussler and
The first Web-enabled version of the system was developed
using HTTP-CL Web server. The current version uses AllegroServe, a Web server
that comes with ACL.
The architecture of the Web-enabled version of SQL-Tutor
is illustrated in the following figure.
Interface of the Web-enabled SQL-Tutor
Try it out: SQL-Tutor
on the Web
(The Web version has been open to outside users on March 29, 2000. Currently
only available to
SQL-Tutor is now
available through the Addison-Wesley's Web Portal DatabasePlace
Downloading SQL-Tutor
The Windows version used to be available for ftp-ing. 1946 people have downloaded the system in the period May 1999 - January 2001. We stopped supporting the stand-alone version of SQL-Tutor. SQL-Tutor is now available through the Addison-Wesley's Web Portal DatabasePlace
Evaluation studies
First study
The first evaluation study was held in April 1998. We thank Franz Inc. for allowing us to use ACL in a way we were not licensed for; we could not perform the study otherwise. The purpose of the study was threefold:
There were 20 students involved in the study, concurrently enrolled in COSC313, a database course. The students have already learnt about SQL in the lectures (roughly 6 sessions), and had 4 weeks of practical labs, defining SQL statements on several databases implemented in Ingres, before the evaluation study. The study involved students using SQL-Tutor in a 2-hours laboratory session. At the end of the session, the students filled a user questionnaire.
Results are published in the paper: Evaluation of a Constraint-Based Tutor for a Database Language Mitrovic, A. Ohlsson, S. Int. J. on Artificial Intelligence in Education, 10(3-4), 1999, pp. 238-256.
May 1999 study
The second evaluation study was held in May 1999, using the Web version of SQL-Tutor. The purpose of the study was threefold:
The students involved were enrolled in COSC313, a database course. The students had already learnt about SQL in the lectures (roughly 6 sessions), and had 4 weeks of practical labs, defining SQL statements on several databases implemented in Ingres, before the evaluation study. The study involved students using SQL-Tutor in a 2-hours laboratory session.
Results are described in the paper: Mitrovic, A., Martin, B. & Mayo, M. Using evaluation to shape ITS design: Results and Experiences with SQL-Tutor. Int. J. User Modeling and User-Adapted Interaction, vol. 12, no. 2-3, pp. 243-279, 2002. .
October 1999 study
The second evaluation study was held in October 1999, using the Web version of SQL-Tutor. The purpose of the study was threefold:
The students involved were concurrently enrolled in COSC205. The study involved students using SQL-Tutor in a 2-hours laboratory session. Results are published in the following papers:
Mitrovic, A., Suraweera, P. Evaluating an Animated Pedagogical Agent. Proc. ITS'2000, G. Gauthier, C. Frasson and K. VanLehn (eds), Springer, pp. 73-82, 2000.
Mayo, M., Mitrovic, A. Using a probabilistic student model to control problem difficulty. Proc. ITS'2000, G. Gauthier, C. Frasson and K. VanLehn (eds), Springer, pp. 524-533, 2000.
2000 study
The goal of this study was to investigate students' self-assessment skills. The study is described in the paper: Mitrovic, A. Investigating students' self-assessment skills. In: M. Bauer, P.J.Gmytrasiewicz and J. Vassileva (eds) Proc. 8th Int. Conference on User Modeling UM-2001, Sonthofen, July 2001, Springer-Verlag LNAI 2109, pp. 247-250.
2001 study
The goal of this study was to investigate the effects of a
simple open student model on students' learning and self-assessment skills.
The study is decribed in a paper
presented at the international conference on Adaptive Hypermedia and adaptive
Web-based systems AH 2002 (
2002 study
The goal of this study was to investigate how the scaffolding
(and fading) effects students' problem-selection skills. The results are
available in the paper presented at the 11th international conference on
Artificial Intelligence in Education, AIED 2003 (
2003 study
The goal of this study was to investigate the effect of an adaptive problem selection strategy on students' learning. The results are described in a paper presented at the Adaptive Hypermedia conference (AH 2004)
2004 and 2005 study
The goal of these two studies was to investigate the effect of
feedback generality level. The results of the 2004 study were presented at
the AIED 2005 conference in
2006 study
In 2006, we investigated the effects of problem templates on learning - the results were presented at AIED 2007: Mathews, M., Mitrovic, A. (Investigating the Effectiveness of Problem Templates on Learning in Intelligent Tutoring Systems. R. Luckin, K. Koedinger, J. Greer (eds) Proc. 13th Int. Conf. Artificial Intelligence in Education AIED 2007, Los Angeles, 2007: 611-613.
2007 study
In 2007, we explored with adding postivie feedback (i.e. feedback on correct actions) to SQl-Tutor. The results of the study show that positive feedback is useful for learning. The paper is coming soon.
Mail all questions regarding SQL-Tutor to sql-tutor@cosc.canterbury.ac.nz
Disclaimer: The opinions expressed in this document are those of
the author, not the
Send mail to Tanja
Mitrovic with questions or comments about this web site.
Copyright © 2007 Intelligent Computer Tutoring Group
Last modified: October 11, 2007