3.3 Frontend-backend communication
3.3.3 POST translation of a sentence
Clicking on a sentence generates a POST request with parameters containing the sentence, gender and language. These three parameters guarantee that the translated sentence from French is translated in the language of the patient with the corresponding gender. The audio file is received as well if it exists. For the sign language video should be returned instead of the translation and the audio. This is not implemented in MobileDr, however it is present in BabelDr.
All these data are fetched from the server with the URL https://regulus.unige.ch/babelDrBidibackend/api/textVersion/translate (see Image 32).
Concerning the gender, male is registered as “default” and female as “female” in the backend.
Image 32 – POST request to send sentence, language, gender and fetch translation and audio from backend
67 3.4 Examples from the code
In the following sections, examples of two main functionalities are presented with snippets of the code. The first is related to the GET request we have seen in Section 3.3.1 to load the domains of the main webpage. The second is about how the chosen answer to a sentence will appear next to that sentence in the list of sentences of a particular subdomain.
3.4.1 Menu with domains
Domains and subdomains are stored in tStorageService. They need to be placed in categories which are Accueil, Spécialités and Finalisation. The COVID-19 domain is a special case.
Since it is in no category, it is removed from the domain list and put into a special variable called specialDomain (see Image 33).
Image 33 – In tDomainSelectionController.js: domains and subdomains saved and put in categories
After successful login to MobileDr and selecting the gender and language of the patient, the domains and subdomains are loaded by the previously explained GET request. This request is defined in tDomainSelectionService as the function getDomainsFromBackend that will load the domains and subdomains from the backend through the appropriate URL (see Image 34).
68 Image 34 – In tDomainSelectionService.js: variable declared contains function, requests domains from backend
It is verified whether this request has already been executed or if it is the first time. If the list of domains stored in tStorageService is empty (length equals zero) the function getDomainsFromBackend is called again from tDomainSelectionService and all subdomains are put in a Map (see Image 35). If it is not empty (length greater than zero) domains will just be put into categories. A Map is a data structure mapping values to keys. The values can then be accessed directly by their key without iterating over the entire data collection to find a required item. In our case the subdomainId is used as a key. The mapping procedure is as follows.
After fetching the domains from the server, it is necessary to extract subdomains from the result set and store them in the Map. Subdomains are grouped by domains. By iterating over domains the corresponding subdomains are extracted for every domain and each subdomain is stored in the Map by calling subdomains.set(subdomain.id, subdomain) where the subdomain object is mapped to its id.
Image 35 – In tDomainSelectionController.js: if domains have never been fetched, fetch them from backend
69 The path to the image of the domain has to be assigned for every domain. The images are stored with the id of the corresponding domain in the filename, thus the path to the domain image can be produced by concatenation, as illustrated at the top of Image 36.
Image 36 – In tDomainSelectionController.js: domains
A domain can contain one or more subjects (for example Cardio + Pneumo) separated by “;”
and all those names have to be collected and stored in a subject variable. The images are linked with the URL depending on the domains id. If no category exists yet, it is created. With the function push() all categories are added one after the other and all domains are loaded (see Image 37).
70 Image 37 – In tDomainSelectionController.js: domains are categorized in an array and loaded
Since the information about the relations between domains and categories is not included in the response from the backend server, we need to provide a static mapping. The mapping is based on the domain id (id=1 categories[1], id=2 categories[2], etc.). We can perform direct mapping (id to array index) because the domain ids are values ranging from 1 to 17.
This is a very limited approach because for any change performed in the backend we need to do the appropriate changes in the frontend. It might be considered to include the category field in the response coming from the backend for future improvements.
The content of the webpage is created by iteration over the categories with ng-repeat directive (see Image 38). In every iteration the title is added to each category and its corresponding domains (items). The <hr> tag displays the horizontal line which underlines each category.
Iterating over the domains displays each domain image (icon) and corresponding domain title under its category. The directive ng-click defines the action to invoke after clicking on the domain image. This includes setting the clicked domain as “selected” in the tStorageService and assigns the path to the corresponding subdomain page. COVID-19 domain is treated separately but in the same way. It also has an image and a text but it is not part of any category.
71 Image 38 – In tDomainSelection.html: creation of classes according to the selected answer to a sentence
3.4.2 Answer in list of sentences
This section describes how the answer to a sentence is handled. As an example, the user goes to the domain Administratif/Anamnèse sociale [Administrative/Social anamnesis] and subdomain Bonjour/Présentation [Hello/Presentation], and chooses the question savez-vous lire ? [can you read?] It is a sentence which has the type of answer oui/non/ne sait pas [yes/no/does not know]. The user selects the answer Oui [yes]. This button is displayed in the following HTML code where the ng-click is defined (see Image 39), meaning the click on this button will call the answerQuestion() function with parameter “yes” in the corresponding controller.
Image 39 – In tTranslation.html: button “yes” declared and when clicking on it something will happen
72 The answerQuestion() function accepts the parameter answer (which is “yes”) and this value will be stored in the tStorageService and in the tReportService (see Image 40). In the following step the program navigates to the previous page where there is the list of sentences of the currently selected subdomain.
Image 40 – In tTranslationController.js: answer value stored in tStorageService and tReportService.
In the following code snippet of the file tDomainQuestions.html (see Image 41), ng-repeat iterates over items (the sentences). The sentences are the ones in a selected subdomain. Then, classes are added according to the selected answer. If the answer is “yes”, the added class will be answer-yes, for “no” it will be answer-no, for “?” (“does not know”) answer-unknown class will be added, for “understood” it will be answer-ok. The named answer-answered class will be added if the answer exists and (&&) if the answer is different from the four answers just mentioned (symbolized by the code ![‘yes’, ‘no’, ‘ok’, ‘unknown’] in Image 41).
Image 41 – In tDomainQuestions.html: creation of classes according to the selected answer to a sentence.
73 The answer will also be stored in tStorageService because the system has to remember which answer was selected to which sentence and also in tHistoryModal.js and tReportService.js.
This is to be able to see it in the history and PDF report file. The icon, its color, size, etc. are defined in myApp.css which will be called through the added class (see Image 42).
Image 42 – In myApp.css: position and styling of the “yes” and “ok” answer icons.
3.5 Conclusion
This chapter described the technologies used in the implementation of the MobileDr web application. The overall internal file organization and frontend-backend communication were explained in some detail. Small code snippets were included to demonstrate the programming techniques used. It is hoped that it can contribute to a possible future development.
74
4 EVALUATING BABELDR AND MOBILEDR
This chapter presents the evaluation methodology, results and their analysis. The evaluations are based on unguided uses of the two applications by doctors, followed by completion of two identical generic SUS questionnaires (one for each system) plus one customized SUS questionnaire designed to access usability of the MobileDr features. Section 4.1 describes the participants and Section 4.2 the evaluation scenario. In Section 4.3 we explain the System Usability Scale questionnaires used for the evaluations and how their results are rated. Section 4.4 shows the results (detailed scores) of every participant and question for BabelDr and MobileDr.
In Section 4.5, we analyze the results in relation with our first research question “which web application (BabelDr or MobileDr) is the best suited for use on a mobile device, in terms of ease of use and learning curve?”. Then we take a closer look at the agreement between the participants, in relation with our second research question “do medical practitioners working in different contexts agree on the features and functionalities required for a mobile medical translation application?”.
Section 4.6 presents the results of the customized SUS questionnaire in relation with the MobileDr features. Finally, Section 4.7 presents the possible improvements of MobileDr from the participants’ qualitative point of view.
4.1 Participants
Due to the pandemic situation it was not easy to find doctors who would have enough time to invest in evaluating BabelDr and MobileDr as well as answering the three planned questionnaires. Nonetheless, there was an in-between pandemic waves period were the situation in hospitals was stable for several weeks. The opportunity was taken to contact 8 doctors, 6 of them being affiliated to the HUG and 2 of them working in their own practice or private center. As described in Table 4 below, 3 doctors are part of the SMPR (emergency
75 service), 2 doctors are specialists, 1 doctor is a generalist (family doctor) and 2 doctors are Interns (which means they do not possess the FMH yet).
Table 4 – Presentation of participants (doctors) selected to evaluate BabelDr and MobileDr
Doctors from different medical professions, working at the HUG or not, have been chosen on purpose. Indeed, the aim of the evaluation was not to analyze a specific sample of participants, but rather to obtain the opinion of different medical practitioners to understand how BabelDr and MobileDr are considered from different perspectives. The questionnaires were intended to identify whether there would be a preference tendency towards one application or the other. Therefore, this evaluation has been proposed for a qualitative view in order to identify which aspects of both applications are good enough and which ones would need further improvement.
4.2 Evaluation scenario
The evaluation method was done in person by addressing doctors directly. Since the doctors could not tell in advance how much time they would have for the evaluation, it was not possible to set the same time limit for all of them. An appointment was taken between two consultations, inside or outside the HUG according to their preference and availability, so the doctors also did not have the same evaluation environment. SMPR doctors have already used BabelDr, others have only heard about it. This means that they did not all have the same experience with BabelDr, but none had tested MobileDr before. During the meeting, a very brief description of the BabelDr project and its purpose was given to introduce the subject to the doctors, which was appreciated.
76 Neither tutorial video, nor any specific information were sent in advance to the participants for testing the applications. The idea was to have them evaluate the intuitiveness of the applications. Since our guidelines specified an intuitive user interface, the need to watch a tutorial video before using the applications was considered as not conforming to these guidelines. Indeed, we chose to define the application as intuitive if the user can spontaneously find out how to use it. That is also why no specific instructions were given to the doctors before testing.
During the testing of BabelDr, a rapid explanation had to be given about the three possibilities to find a desired sentence (using the voice recognition tool, searching manually in the set of sentences available within a domain and typing a keyword in the keyword search tool); each doctor tested all three of them. While for MobileDr the participants were not given any guidance to see how well they would be able to navigate naturally by clicking on a domain, a subdomain and the sentence.
The doctors drew which application they would evaluate first, BabelDr or MobileDr. The reason for this was to avoid a comparison bias. Indeed, the second application can always be compared to the first one. Therefore, it is important to randomize which one is evaluated first.
Depending on the result, they tested the corresponding application on a mobile device by going through it spontaneously without much thinking, as it is required before evaluating with the SUS. The first application testing lasted a few minutes; they spontaneously stopped when they felt they had gone through every aspect of the application. They then completed the corresponding SUS questionnaire.
The same was done with the second application by spontaneous discovery and the second corresponding questionnaire was completed (both questionnaires contained exactly the same questions). After that, the third customized SUS questionnaire concerning exclusively MobileDr was completed and a small debriefing took place during which the doctors expressed their individual opinions concerning each application and provided feedback. Now follows an explanation about the System Usability Scale (SUS).
77 4.3 System Usability Scale (SUS)
The System Usability Scale (SUS) (Bangor, Kortum, & Miller, 2008) is the one “quick and dirty” reliable method used to evaluate, as its name indicates, the usability of a system (here both applications). It is made up of ten general questions, each one assessing an aspect of the evaluated system such as the need for support, training and complexity. The respondent has to answer each question on a 1 to 5 scale, where position 1 refers to a “strongly disagree” feeling and position 5 to “strongly agree”.
The questions alternate between positive and negative sentence formulation, forcing the respondent to think twice before giving an answer. SUS does not target users who would have trained intensively to use the system, but rather those who skim through it fairly quickly and spontaneously. They then answer the questionnaire without thinking too much about each aspect, so that their first overall impression is reflected.
See appendix A which shows the SUS filled by the eight doctors who took part in the evaluation of both MobileDr and BabelDr to have a precise idea of the 10 questions. All evaluations have been conducted in French, since most doctors are French native speakers.
Appendix C presents the French translation of this SUS that was submitted to the participants.
The answer of each question receives a grading depending on its positive or negative formulation. Positive questions get a grade from 0 (corresponding to position 1 “strongly disagree”), to 4 (corresponding to position 5 “strongly agree”) and negative questions get the opposite grading (grade 0 for position 5 “strongly agree”, to grade 5 for position 1 “strongly disagree”). Figure 1 shows more concretely how the grading happens:
I thought the system was easy to use → positively formulated question
Strongly disagree Strongly agree
1 2 3 4 5
Grades: 0 1 2 3 4
I found the system very cumbersome to use → negatively formulated question
Strongly disagree Strongly agree
1 2 3 4 5
Grades: 4 3 2 1 0
Figure 1 – The attribution of grades depending on the formulation of the question
78 A third questionnaire was given in addition (see appendix B, as well as appendix D for its French translation). It is a customized SUS where half of the questions are also positive and half negative, but not alternating (just randomly put). It was thought useful to randomize the order so that if the doctors noticed the alternation of questions in previous questionnaires, they would not follow the same alternating responses.
Some doctors were a bit destabilized indeed, not only by the questions being sometimes positive and sometimes negative, but also their order. If there was no repetition in the pattern, they seemed to read the questions much more carefully. This assured that the given answers were really thought upon. Therefore, it can be stated that their responding was sincere and accurate.
The ten sentences of the SUS are the following:
1. I think that I would like to use this system frequently 2. I found the system unnecessarily complex
3. I thought the system was easy to use
4. I think that I would need the support of a technical person to be able to use this system 5. I found the various functions in this system were well integrated
6. I thought there was too much inconsistency in this system
7. I would like imagine that most people would learn to use this system very quickly 8. I found the system very cumbersome to use
9. I felt very confident using the system
10. I needed to learn a lot of things before I could get going with this system
These sentences have been considered as questions, since they triggered an answer coming from the participants. This is how they will be called in the following sections. The scores of these questions will now be presented.
79 (doctors) are written as D1 to D8 and the ten questions as Q1 to Q10.
D1 D2 D3 D4 D5 D6 D7 D8 tot /32
Table 5 – BabelDr individual SUS scores obtained by all eight doctors (D1-D8) in all ten questions (Q1-Q10)
D1 D2 D3 D4 D5 D6 D7 D8 tot /32
Table 6 – MobileDr individual SUS scores obtained by all eight doctors (D1-D8) in all ten questions (Q1-Q10)
80 To have a visual overview, the five possible scores are represented and interpreted as follows:
score 4 dark green – very good
score 3 light green – good
score 2 light blue – neutral or average
score 1 light red – poor
score 0 dark red – insufficient
The side tables on the bottom (“tot/40”) show the total grades per doctor, obtained by summing up the grades for each doctor (vertical lines). The maximum possible score for each doctor is 40, obtained when multiplying 10 questions by the maximum score of 4. This gives an overview of the overall rating by each individual doctor. The rating per doctor will be used in the qualitative approach comparison of BabelDr and MobileDr.
The side tables on the right (“tot/32”) show the total grades per question, obtained by summing up the grades for each question (horizontal lines). The maximum possible score for each question is 32, obtained when multiplying 8 doctors by the maximum score of 4. This gives an overview of the overall rating of each individual question. The rating per question will be used in particular in the quantitative approach comparison of BabelDr and MobileDr.
When comparing both “tot/32” and “tot/40” side tables of Table 5 (BabelDr) and Table 6 (MobileDr), it appears that all doctors gave higher scores to MobileDr and that all questions were rated higher for MobileDr. This will be discussed by the following quantitative analysis.
4.5 Quantitative analysis
First, we present a non-statistical comparison of the results of Table 5 (BabelDr) and Table 6 (MobileDr) based on the doctors grouped by their service affiliation to see if there are differences in opinions for each group. Then, from a more statistical point of view, both applications are compared per question, as well as on a global level.
81 4.5.1 Service rating comparison
As presented in Table 4, the participants work in different services. D1, D2 and D3 are in SMPR, D4 and D5 are in Interns, D7 is a generalist and D6 and D8 are specialists. These doctors grouped by their service rated both applications as follows.
total: normalized:
SMPR rated BabelDr with: 29/40, 24/40, 33/40 86/120 72/100
Interns rated BabelDr with: 16/40, 21/40 37/80 46/100
Generalist rated BabelDr with: 28/40 28/40 70/100
Specialists rated BabelDr with: 9/40, 19/40 28/80 35/100 SMPR rated MobileDr with: 31/40, 39/40, 38/40 108/120 90/100 Interns rated MobileDr with: 35/40, 37/40 72/80 90/100
Generalist rated MobileDr with: 38/40 38/40 95/100
Specialists rated MobileDr with: 34/40, 39/40 73/80 91/100
For the sake of readability, in the following discussion the masculine form is used to refer to
For the sake of readability, in the following discussion the masculine form is used to refer to