5.3 Criteria and protocol for the design of exploratory search tasks
5.3.4 Protocol for the design of exploratory search task
As a reminder, we design this elaborating exploratory search task protocol for com-puter scientists who are both the designers and the evaluators of a given exploratory search system. We want a step by step protocol, as simple as possible. With this goal in mind, the keywords and the activities exposed in these kits of Bloom’s taxonomy are really useful because they can be used for the design of the search goal. From the kit of Bloom’s taxonomy, we use the keywords, the possible outcomes and the actions.
From the kit, we list these relevant terms which can be used in our protocol. For example, in the kit the different outcomes inspired us to propose our own list of outcomes that could be used by the evaluators in their task design: (make a) list, presentation, demonstration, plan, survey, examples, summary. . . In step 2.2 we ask them to select one outcome that the user will do in her search session.
In the following Table 5.1 we present our protocol of designing exploratory search tasks. We add in parallel an example of the outcome obtained for the exploratory search system Discovery Hub. The protocol is divided into two parts: first, the elaboration of the task (topic, goal and context design) and secondly the verification that the elaborated task is ecological and respects the characteristics we proposed in Section 5.2.
We use the keywords, actions and outcomes of the Bloom’s taxonomy kit in the step 2.2 which consists of defining for what purpose the user will do an exploratory search task: discoverthe topic, find resultssimilarto the topic, make alist/plan/ presentation/ paper /report, prepare oforganizean event or a trip, giveexample, comparetwo different concepts,findinformation. . . This list can be used or adapted by the evaluators as they like.
Protocol’s steps Outcome example for Discovery Hub
STEP 1:Define a topic to explore by the users on your exploratory search system.The used database and/or the goal of your system can guide you in this topic definition.
Discovery Hub is based on DBpe-dia (a open knowledge graph of Wikipedia developed by the Seman-tic Web community:https://wiki.
dbpedia.org/about/). With this system, a user can explore a wide range of subjects (countries, music, politics. . . ). In this example we choose the topicSenegal.
STEP 2: In this step, you will define the goal of the exploratory search task
STEP 2.1:Indicate why the user will explore the selected topic. Choose one of the these activities :
• Elaborate something (e.g. a text, a pre-sentation, a plan. . . ).
• Learn more about the subject, linked to the users’ personal or professional interests.
• Share information on the subject in or-der to: recommend, advice, teach, ex-plain, etc.
STEP 2.1: Learn more about the Senegal
STEP 2.2: Indicate what the user will do to achieve her goal. For example: discover the topic, find results similar to the topic, make a list / plan / presentation / paper / report, prepare of organize an event or a trip, make a diagram, give example, compare two different concepts, find information on the selected topic (e.g. bookmark results), compare. . .
STEP 2.2: make a list
Continued on next page
Tab. 5.1 –Continued from previous page
Protocol’s steps Example with Discovery hub
STEP 2.3: From the previous elements, write down the task and give a context of search. Ask yourself why the user will do such an exploration. This context will al-low an easier identification and appropri-ation of the exploratory search task. It has to be plausible and realistic with regards to the user’s life and habits.
STEP 2.3:"A friend from Senegal in-vited you to visit him in Dakar. To prepare your trip, you want to list the interesting things to see and dis-cover"
STEP 3: Make sure that the task isecological.
In other words, the task should represent a real-life situation or a simulation of daily ac-tivities. The explored topic should suit the user’s interest and/or profession (e.g. “write an article on a topic” should be right only for journalists or bloggers because it is in line with their daily activity and profession).
STEP 3:It is OK. Our task written in step 2.3 presents an invitation from a friend. It is a common event which can concern anybody.
STEP 4: Make sure that the task won’t con-tain clear criteria on when to end the search.
For example, do not provide limits for the du-ration of the search session and the number of results the user have to find ("find three Ital-ian restaurants" vs. "find ItalItal-ian restaurants").
You only have to give a task and explain why the user have to do it.
STEP 4:The task is ecological. There is no time limit or specified number of results requested for the list.
STEP 5: Make sure that the specification on how to find the information or how to eval-uate them is kept to the minimum, or not specified at all. In this sense, the user should not have a clear idea on how to begin the search.
STEP 5: This is indeed the case.
We did not specified what to search (places, food, museums, event. . . ).
The user may interpret and appropri-ate the task.
Tab. 5.1: Protocol for the design of exploratory search tasks