A Self-efficacy Approach to Assess the Effectiveness of Web Localisation and Accessibility Training

(1)

Conference Presentation

Reference

RODRIGUEZ VAZQUEZ, Silvia, O'BRIEN, Sharon

Abstract

In order to ensure that multilingual web content is accessible to everyone, including those with functional diversity, we need to encourage an 'accessibility thinking' among future localisation professionals. However, to the best of our knowledge, most training institutions do not teach yet the needed technical competence on the matter. Dublin City University in Ireland and the University of Geneva in Switzerland were two of the few exceptions in the last two years, as they offered a Master's course on localisation which included the acquisition of basic accessibility knowledge and know-how as one of the main learning outcomes. With a view to assessing the effectiveness of such an accessibility-enhanced learning programme, we conducted an exploratory study with the postgraduate students enrolled in the module, at both universities. By adopting an action research strategy, we administered a scientifically validated questionnaire built around the concept of self-efficacy at the beginning of the web localisation module (week 6 of the course) and at the end (week 9). As described in prior work (Doherty and Kenny, 2014), the [...]

RODRIGUEZ VAZQUEZ, Silvia, O'BRIEN, Sharon. A Self-efficacy Approach to Assess the Effectiveness of Web Localisation and Accessibility Training. In: 1st International

Conference on Translation and Cultural Sustainability: Foundations, Fundamentals and Applications, Salamanca (Spain), 28-30 November 2018, 2018

Available at:

http://archive-ouverte.unige.ch/unige:112485

Disclaimer: layout of this document may differ from the published version.

1 / 1

(2)

A Self-efficacy Approach to Assess the Effectiveness of Web Localisation and

Accessibility Training

Silvia Rodríguez Vázquez & Sharon O’Brien

silvia.rodriguez@unige.ch · sharon.obrien@dcu.ie

Salamanca, Spain 28-30 December 2018

(3)

AGENDA

?

Research Context

 How we understand web localisation

 Web accessibility (WA) and prior work on multilingual WA

Research Motivation

Methodological Framework

Research Indicators & Future Research Avenues

Main Findings

(4)

“The aim of localisation should be that people from a specified locale can use the product without any difficulty in their own language”

Same language

Same cultural conventions Not necessarily same physical

and sensory abilities

Ensuring linguistic & cultural adequacy Removing functionality-related obstacles

Sandrini (2008)

Research Context

Web Localisation

(5)

Web Accessibility Perspectives

“Essential for some, useful for all”

“All users (especially those with disabilities) can perceive, understand, navigate, interact with and contribute to the web”

Dpt. Social Services, Australian Government

Web Accessibility

Research Context

Web Accessibility

(6)

Perceivable

1.1 Text Alternatives:

Provide text alternatives for any non-text content (changeable to large print, braille, speech, symbols or simpler language...)

1.2 Time-based Media:

Provide alternatives for time-based media.

1.3 Adaptable:

Create content that can be presented in different/simpler ways without losing information or structure.

1.4 Distinguishable:

Make it easier for users to see and hear content including separating foreground from background.

Operable

2.1 Keyboard Accessible:

Make all functionality available from a keyboard. 2.2 Enough Time:

Provide users enough time to read and use content.

2.3 Seizures:

Do not design content in a way that is known to cause seizures.

2.4 Navigable:

Provide ways to help users navigate, find content, and determine where they are.

2.5 Input Modalities:

Make it easier for users to operate functionality through various inputs beyond keyboard.

Understandable

3.1 Readable:

Make text content readable and understandable.

3.2 Predictable:

Make Web pages appear and operate in predictable ways.

3.3 Input Assistance:

Help users avoid and correct mistakes.

Robust

4.1 Compatible:

Maximize compatibility with current and future user agents, including assistive technologies.

W C A G 2 . 1 P r i n c i p l e s a n d g u i d e l i n e s

C h a l l e n g e 1 . Tr a n s f er a b i l i t y

Research Context

Multilingual Web

Accessibility

(7)

Webmasters Content authors

Translators

Localisation engineers Testers

Monolingual Multilingual

Multiple locales Translation and

localisation tools

Localisation standards (XLIFF, ITS, TMX…)

Research Context

Multilingual Web Accessibility

Ch a l l e n g e 2 . U n i v e r s a l i t y o f

t h e WA mo d el

Image: Adaptation by the authors.

Original Image Credits:

Image by Michael Duffy, from: Essential Components of Web Accessibility. S.L. Henry, ed. W3C (MIT, ERCIM, Keio). Status:

Updated August 2005.

https://www.w3.org/WAI/fundamentals/co mponents/Last access: December 2018.

(8)

2 0 1 2 - 2 0 1 8

R o l e o f w e b l o c a l i s a t i o n p r o f e s s i o n a l s i n t h e a c h i e v e m e n t o f a n a c c e s s i b l e m u l t i l i n g u a l w e b

Questionnaires

Analysis of web documents

145 subjects

(web accessibility experts, academics, localisation professionals

and other localisation industry representatives, screen reader users)

15,000 web pages Interviews Tool review &

creation Controlled Experiments

 Lack of awareness / low level of awareness

 Insufficient knowledge

 Uncertainty regarding accountability for MWA

 Lack of support in translation technology

Research Context

Prior Work

(9)

 Ever-evolving industry

 Technological progress

 Increasing language coverage

 New client demands

 Strengthened service portfolio

 LSPs now see themselves as

“Solutions companies”, “one stop shops”

Multilingual SEO

Digital Marking Strategy Social media

Globalisation and internationalisation

Multimedia

Usability testing

Research Motivation

1) Favourable

localisation landscape

(10)

 Most training institutions do not teach yet the needed theoretical and technical competence on the matter

 University of Salamanca, 2013 - Present

 University of Geneva, Switzerland, 2013 - Present – cf. Rodríguez Vázquez (2014)

 Dublin City University, Ireland (2016-2017)

 Accessibility is not yet fully integrated in localisation courses

 Studied as a separated module within the course

 Effectiveness not measured yet quantitatively

Research Motivation

2) Lack of WA training in Translation curriculum

(11)

Research Motivation

Torres del Rey & Rodríguez Vázquez (2016)

3) Development of a

theoretical framework

( w o r k i n p r o g r e s s )

 ALMA project

 (Análisis y adecuación de los

fundamentos e instrumentos de la Localización para la Mejora de la Accesibilidad en contenidos web)

 Localiser competence proposal

 HCI-related sub-competences

 Accessibility, usability, UX…

(12)

Goal: Measure the effectiveness of a learning

programme offered to localisation students on the topic of web localisation and accessibility

The study

Overview

2016/2017 2017/2018

 MSc/MA Translation Technology

 Localisation course

 Web localisation module: 3 weeks (4 lecture hours and 2 lab hours)

 HTML & CSS

 Web localisation best practices

 Web accessibility best practices

Procedure

• Presentation of the study

• Consent form

***

1. Questionnaire 1 (before web l10n module)

2. Web localisation module

3. Questionnaire 2 (after web l10n

module)

(13)

Methodology

Measuring effectiveness through self-efficacy

 Questionnaire built around the concept of self-efficacy

(Compeau and Higgings, 1995)

 Individual’s confidence in his ability to control thoughts, feelings and actions to achieve a desired outcome in a given concept.

 Used in different technological contexts in the past

(Joo, Bong and Choi, 2000; Doherty and Kenny, 2014)

 12 different localisation and accessibility-related tasks (6 HTML, 6 WA)

 Task for students:

1. Judge whether the skills they had could help them perform those tasks (Yes/No) 2. Indicate their degree of confidence in their response (10-point scale), where:

• 1: Not at all confident

• 5: Moderately confident, and

• 10: Totally confident

(14)

Methodology

The notion of self-efficacy

Some examples

(15)

Methodology

Self-efficacy level variations

BEFORE AFTER

6h-training Web l10n-a11y

For each task, 12 possible variations:

1. No  Yes, confidence 2. No  Yes, = confidence 3. No  Yes, confidence

4. No  No, confidence 5. No  No, = confidence 6. No  No, confidence

7. Yes  No, confidence 8. Yes  No, = confidence 9. Yes  No, confidence

10. Yes  Yes, confidence

11. Yes  Yes, = confidence

12. Yes  Yes, confidence

(16)

Methodology

Self-efficacy level variations

BEFORE AFTER

6h-training Web l10n-a11y

For each task, 12 possible variations:

1. No  Yes, confidence 2. No  Yes, = confidence 3. No  Yes, confidence

4. No  No, confidence 5. No  No, = confidence 6. No  No, confidence

7. Yes  No, confidence 8. Yes  No, = confidence 9. Yes  No, confidence

10. Yes  Yes, confidence

11. Yes  Yes, = confidence

12. Yes  Yes, confidence

IDEAL

(17)

Participants

8 participants

(age x̄=25.85, sd=1.79 )

23 participants

(age, x̄=25.22, sd=0.67) _0% ^9%

22% 35% 35%

0% 0% 13% 0% 88%

N O N E 1 T O 1 0 1 1 T O 1 9 2 0 T O 2 9 3 0 +

UNIGE DCU

Use of the Web (h/week)

(18)

Main Findings

Self-efficacy (judgement only)

46.77%

11.29%

53.23%

88.71%

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

100.00%

Before After Before After

No Yes

A11Y HTML

Students judgement of what they can do has significantly increased

(p < 0.001)

No Yes

18.82%

1.08%

81.18%

98.92%

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

100.00%

Before After Before After

NoNo YesYes

(19)

32.18% 38.10%

32.32%

4.24%

24.14%

38.10%

34.34%

43.64%

43.68%

23.81%

33.33%

52.12%

B EF O RE AF TER B EF O RE AF TER

NO Y ES

Not at all Moderate Full

Main Findings

Self-efficacy (overall)

A11Y HTML

No Yes

37.14%

50.00%

13.25%

3.26%

22.86% 0.00%

42.38%

29.35%

40.00%

50.00% 44.37%

67.39%

B EF O RE AF TER B EF O RE AF TER

NO Y ES

Not at all Moderate Full

No Yes

Students overall

confidence in being able to carry out a given task has

significantly increased

(p < 0.001)

(20)

Main Findings

Self-efficacy (overall distribution)

Before

After

Before

After

YES NO

ACCESSIBILITYHTML

(21)

Main Findings

Self-efficacy variations (I)

66%

28%

5% 1%

Yes-Yes No-Yes No-No Yes-No

HTML A11Y ALL

Total % Total % Total

Confidence

increase 98 39.84 71 28.86 169

Confidence

stagnation 27 10.98 17 6.91 44

Confidence

decrease 25 10.16 8 3.25 33

(22)

Main Findings

Self-efficacy variations (II)

66%

28%

5% 1%

Yes-Yes No-Yes No-No Yes-No

HTML A11Y ALL

Total % Total % Total

Confidence

increase 14 13.59 27 26.21 41

Confidence

stagnation 4 3.88 4 3.88 8

Confidence

decrease 16 15.53 38 36.89 54

(23)

Research Indicators

Overall…

 Data shows positive learning outcomes

 Significant increase observed in HTML and WA self-efficacy

BUT:

 We need to adopt a critical angle:

 Is a positive judgement with a low or moderate confidence level enough?

 Interpretation of self-efficacy stagnation: Does this mean that we need further training hours? Changes in our training techniques?

 More in-depth analysis needed

Acceptable level of effectiveness of our training programme on

localisation and accessibility

(24)

Future Research Avenues

 Data from the study itself

 Explore correlations between self-rated knowledge of HTML/accessibility and self- efficacy levels

 Explore correlations between self-efficacy levels and final course grades

 In Doherty and Kenny (2014), none was found

 Individual consultation with students

 Integration of accessibility concerns in localisation and translation technology courses

 Continue developing theoretical framework

(25)

Thank you

Silvia Rodríguez Vázquez & Sharon O’Brien

silvia.rodriguez@unige.ch

·

sharon.obrien@dcu.ie

(26)

References

Compeau, Deborah R., and Christopher A. Higgings. 1995. ‘Computer Self-Efficacy: Development of a Measure and Initial Test’.

MIS Quarterly19 (2): 189–211. doi:10.2307/249688.

Doherty, Stephen, and Dorothy Kenny. 2014. ‘The Design and Evaluation of a Statistical Machine Translation Syllabus for Translation Students’. The Interpreter and Translator Trainer8 (2): 295–315. doi:10.1080/1750399X.2014.937571.

Jiménez Crespo, Miguel Ángel. 2013. Translation and Web Localization. London: Routledge.

Joo, Young-Ju, Mimi Bong, and Ha-Jeen Choi. 2000. ‘Self-Efficacy for Self-Regulated Learning, Academic Self-Efficacy, and Internet Self-Efficacy in Web-Based Instruction’. Educational Technology Research and Development 48 (2): 5–17.

doi:10.1007/BF02313398.

Rodríguez Vázquez, Silvia. 2014. ‘Introducing Web Accessibility to Localization Students: Implications for a Universal Web’. In Proceedings of the 16th International ACM SIGACCESS Conference on Computers & Accessibility, 333–334. ASSETS ’14. New York, NY, USA: ACM. doi:10.1145/2661334.2661414.

Sandrini, Peter. 2008. ‘Localization and Translation’. Edited by Heidrun Gerzymisch-Arbogast, Gerhard Budin, and Gertrud Hofer.

MuTra Journal, LSP Translation Scenarios. Selected Contributions to the EU Marie Curie Conference, Vienna 2007, 2: 167–91.

Torres del Rey, Jesús, and Silvia Rodríguez Vázquez. 2016. ‘New Insights into Translation-Oriented, Technology-Intensive Localiser Education: Accessibility as an Opportunity’. In Proceedings of the 4th International Conference Technological Ecosystems for Enhancing Multiculturality (TEEM ’16), 971–78. New York: ACM Press.

doi:https://doi.org/10.1145/3012430.3012634.

(27)

Additional slides

(28)

HTML

H T M L -r e l a t ed

s t a t emen t s

(29)