DMLL: ML journal Special issue on Data Mining Lessons Learned

[Nada Lavrac]

Machine Learning Journal:
Special Issue on Data Mining Lessons Learned
http://www.hpl.hp.com/personal/Tom_Fawcett/DMLL-MLJ-CFP.html

Guest editors:
Nada Lavrac, Hiroshi Motoda and Tom Fawcett

Submission deadline: Monday, 7 April, 2003.

Call for Papers

Data mining is concerned with finding interesting or valuable patterns
in data. Many techniques have emerged for analyzing and visualizing
large volumes of data, and what we see in the technical literature are
mostly success stories of these techniques. We rarely hear of steps
leading to success, failed attempts, or critical representation
choices made; and rarely do papers include expert evaluations of
achieved results. Insightful analyses of successful and unsuccessful
applications are crucial for increasing our understanding of machine
learning techniques and their limitations.

Challenge problems (such as the KDD Cup, COIL and PTE challenges) have
become popular in recent years and have attracted numerous
participants. These challenge problems usually involve a single
difficult problem domain, and participants are evaluated by how well
their entries satisfy a domain expert. The results of such challenges
can be a useful source of feedback to the research community.

At ICML-2002 a workshop on Data Mining Lessons Learned was held and
(http://www.hpl.hp.com/personal/Tom_Fawcett/DMLL-workshop.html) and
was well attended. This special issue of the Machine Learning journal
follows the main goals of that workshop, which are to gather
experience from successful and unsuccessful data mining endeavors, and
to extract the lessons learned from them.

Goals

The aim of this special issue is to collect the experience gained from
data mining applications and challenge competitions. We are interested
in lessons learned both from successes and from failures. Authors are
invited to report on experiences with challenge problems, experiences
in engineering representations for practical problems, and in
interacting with experts evaluating solutions. We are also interested
in why some particular solutions - despite good performance - were
not used in practice, or required additional treatment before they
could be used.

An ideal contribution to this special issue would describe in
sufficient detail one problem domain, either an application or a
challenge problem. Contributions not desired for this special issue
would be papers that report on marginal improvement over existing
methods using artificial synthetic data or UCI data involving no
expert evaluation. We offer the following content guidelines to
authors.

1. For applications studies, we expect a description of the attempts
that succeeded or failed, an analysis of the success or failure, and
any steps that had to be taken to make the results practically useful
(if they were). Ideally an article should support lessons with
evidence, experimental or otherwise; and the lessons should generalize
to a class of problems.

2. For challenge problems, we will accept either experiences preparing
an individual entry or an analysis of a collection of entries. A
collective study might analyze factors such as the features of
successful approaches that made them appealing to experts. As with
applications studies, such articles should support lessons with
evidence, and preferably should generalize to a class of
problems. Analyses should preferably shed light on why a certain class
of method is best applicable to the type of problem addressed.

3. A submission may analyze methodological aspects from individual
developments, or may analyze a subfield of machine learning or a set
of data mining methods to uncover important and unknown properties of
a class of methods or a field as a whole. Again, a paper should
support lessons learned with appropriate evidence. 

We emphasize that articles to appear in this special issue must
satisfy the high standards of the Machine Learning journal.
Submissions will be evaluated on the following criteria:

Novelty: How original is this lesson? Is this the first time this
observation has been made, or has it appeared before?

Generality: How widely applicable are the observations or conclusions
made by this paper? Are they specific to a single project, a single
domain, a class of domains, or much of data mining?

Significance: How important are the lessons learned? Are they
actionable? To what extent could they influence the directions of work
in data mining?

Support: How strong is the experimental evidence? Are the lessons
drawn from a single project, a group of projects, or a thread of work
in the community?

Clarity: How clear is the paper? How clearly are the lessons
expressed?

The criteria for novelty, significance and clarity apply not only to
the lessons but also to the paper as a whole.

Submission Instructions

Manuscripts for submission should be prepared according to the
instructions at http://www.cs.ualberta.ca/~holte/mlj/

In preparing submissions, authors should follow the standard
instructions for the Machine Learning journal at
http://www.cs.ualberta.ca/~holte/mlj/initialsubmission.pdf

Submissions should be sent via email to Hiroshi Motoda
(motoda at ar.sanken.osaka-u.ac.jp), as well as to Kluwer Academic
Publishers (jml at wkap.com). In the email please state very clearly that
the submission is for the special issue on Data Mining Lessons
Learned.