Esta obra está bajo licencia internacional Creative Commons Reconocimiento 4.0.
DEVA una Metodología basada en diseño centrado en el usuario para el desarrollo de un sistema buscador de documentos judiciales
Corresponding Author(s) : Hugo Ordoñez Erazo
Investigación e Innovación en Ingenierías,
Vol. 10 Núm. 1 (2022): Enero-Junio
Objetivo: Definir e implementar los componentes para el desarrollo de un motor de búsqueda de documentos judiciales. Metodología: La metodología consta de tres fases: a) Definición: Incluye una revisión del estado del arte de las tecnologías asociadas al proyecto. b) Desarrollo de prototipos: Aquí se desarrollan los diferentes módulos y componentes del sistema. c) Evaluación: enfocado a la usabilidad y satisfacción del usuario. Resultados: Se identificaron las características y elementos necesarios para el buscador. Para ello se realizó un Benchmarking sobre aspectos generales y componentes de los buscadores de jurisprudencia. Adicionalmente, se realizó una encuesta a 79 expertos legales. Finalmente, se realizó una prueba de uso. Conclusiones: Se diseñó un prototipo del buscador de acuerdo a las características declaradas por los usuarios. Este prototipo mostró una óptima usabilidad y navegación debido a que fue diseñado considerando las necesidades de jurisprudencia de usuarios reales. Sus interfaces son fáciles de entender y sus filtros hacen que el proceso de búsqueda sea más ágil y preciso. Finalmente, el motor de búsqueda desarrollado ofrece una buena experiencia a los usuarios.
In Colombia, the capacity of judges to make decisions and interpret the rules is fundamental in the judicial function. The judges not only limit themselves to the application of current norms, but also resort to justifications of their own reasoning. Thus, it is necessary to impose limits that guarantee legal security and equality in access to the administration of justice, so that individuals are not at the mercy of the subjectivity of judges. One of these tools is the judicial precedent. However, finding the judicial precedent for a specific case may become a cumbersome task. This search involves thoroughly reviewing a large number of jurisprudential texts to identify arguments in favor of their interests 1.
Currently, there are some jurisprudence search engines. However, these platforms do not offer effective solutions for the needs of the end user. Therefore, it becomes difficult to access the jurisprudence information easily and quickly. Here we propose a methodology called definition, development and evaluation (DEVA). DEVA aims to design a search system for judicial documents, which facilitates the search for jurisprudential documents by following a series of steps dictated by users and experts in web application design, where at the end different evaluations of user satisfaction are carried out to determine if the search engine developed under the methodology is better than the search engines used for this purpose today. The rest of the article is organized as follows: Section 2 describes the related work. Section 3 introduces DEVA. Section 4 presents the evaluation of the system, and finally, section 5 presents the conclusions and future work.
Jakob Nielsen and Rolf Molich 2 defined usability principles that focus on user behavior when interacting with a web page or computer application. The authors propose qualitative measures regarding the user experience. They argue that for any computer system to be usable, a user-centered methodology must be used, where user’s opinion is the starting point in software development.
The work described in 3 addresses the relationship between usability and user experience through a study conducted on 21 people who evaluated these two concepts in up to 8 software products. Users express their decisions, opinions, and points of view through questionnaires and interviews. This research found that users perceive usability as a great contributor to the user experience.
Thanks to the User-Centered Design (DCU), it has been possible to achieve high-quality standards in products for end-users, 4) proposes a global approach that allows integrating the DCU within various software development models such as the Cascade Model, Iterative Model, and Agile Model explores in a generic way the integration points of the DCU to the Software Development Life Cycle (SDLC), defining them in different stages.
In Peru, a research project was developed for the design of a knowledge-based Environmental Decision Support System (EDSS) that allowed low-tech users to easily use an expert system comprised of databases, analytical processing algorithms, and user interface 5. This EDSS system uses knowledge bases to make decisions on a specific case, facilitating the HCI since the design was adapted to the user´s preferences and their values.
In Colombia 6, a platform based on multicultural design was developed focused on the user for indigenous census, in this platform different points of view are taken to create each of its components taking into account indigenous people and their community as well as different actors who do not belong To the indigenous community, this platform is evaluated with different metrics such as also validated by user satisfaction.
In Colombia, a legal structure has been designed as the formal source of law. Authorities must apply norms and rules with the same factual and legal assumptions on the constitutional court ruling, thus, lawyers must use previous judgments to support their current cases 7. Legal informatics is a set of tools to facilitates the development of activities corresponding to the legal area, such as searching for documents and transcribing information through a machine. 8. In this vein, DCU has the potential to offer users an adequate experience in searching for judicial precedents through the development of different tests and prototypes based on experts.
The proposed methodology consists of 3 phases (see Figure 1): definition, development, evaluation (DEVA)
Figure 1. Phases of the DEVA Methodology. Source: self made
Phase 1. Definition: this phase includes the review of the state-of-the-art on the relevant areas and technologies. Besides, this phase consists of the learning of these technologies for the development of the project. Finally, it was included the identification of the target population.
Initial review of the state-of-the-art. Exploration of tools and technologies.
Phase 2: Development, this phase comprises activities related to developing the experimental prototype, its different modules and other components, using the DCU, and HCI methodology.
User-centered design (UCD): According to ISO 9241-210 of 2010 9, UCD establishes a cyclical model of the decisions making. This process is composed of four phases are proposed, described below (see Figure 2)
Figure. 2 Phases of the cyclical model 10 Source: self made
First, the potential users are identified (User identification), then the primary purpose of the system and its requirements are established (Definition of the use). Then, a design is proposed, seeking to reduce errors in the final product (Design solution). Finally, the system is tested by users, who provide feedback.
Each phase of the cyclical model is composed of activities. For the user identification phase, interviews, questionnaires, focus groups, theoretical and contextual research are used 11. with this information, it is possible to evaluate the following criteria:
User Test Heuristic Evaluation
To requirements capture (Definition of the use), Card-Sorting is used, which is considered an efficient technique to extract the semantic structure of the requirements and their relationship 12.
Non-functional requirements are the aspects of the system that can be perceived by users, but do not have a direct relationship with the functional behavior of the system. These requirements has constraints such as: 13
Response time The precision Security Resources consumed
A first prototype of the system was developed to identify the opinion of the users. Also, and a validation test of HCI was carried out. In this test, elements to improve were identified, the respective modifications were carried out, taking into account these suggestions to ensure that the search engine was really focused on the user needs regarding navigability, visualization of information, and response time14,15,16. This process was repeated iteratively. Finally, a comparative test is subsequently carried out between other jurisprudence search engines shown in figures 3 y 4.
Figure. 3 Search engine of the Relatoria of the Supreme court of Justice. Source: self made
Figure 4. Search engine of the Supreme Court of Justice. Source: self made
Figure 5 shows the initial page of the developed search engine. The interface is simple, and its colors are clear. If this design is compared to that of figures 3 and 4, it is observed that the logo and the colors are different; The wide search bar was added without having to enter another page. Additionally the different filters were added to apply in the search.
Figure. 5 Initial page of Lawyer Source: self made
Figure 6 shows how the search results are displayed. Lawyer is much simpler compared to other search engines. Beside it includes filters that can be applied to order the results by year, title, courts, guardianship, or cars. Additionally, two filters can be applied to the search to improve the results.
Figure 6 Results visualization in Lawyer. Source: self made
Finally, figure 7 depicts the whole sentence, that is, the complete results. Besides it has the option of downloading the sentence.
Figure 7 Jurisprudence document Source: self made
As mentioned before, A comparative test was carried out between search engines. The process used to apply the comparison test and its results are presented below.
Phase Three: Evaluation includes activities related to the evaluation usability and user satisfaction 14.
Nielsen’s Heuristics User’s satisfaction 17
Search engine comparison test
A survey was carried out to determine general information about the participant and its relationship with the jurisprudence searches. During the surveys, 2 people were required (monitor and technical collaborator) to supervise the process and the participant (user). The monitor was responsible for carrying out the test, that is, the one who directed the test and asked the questions18. The technical collaborator was in charge of supervising that the tests had the necessary elements to be performed. He will also carry out the audiovisual record of the survey. For the test, a Laptop with Morae software previously installed and verified was necessary.
During the test, the participants were informed that the objective of the test was to compare different search engines in the Colombian legal environment, namely: “Supreme court search engine”, “judicial branch search engine”, and "Lawyer" search engine. The participants who took part in the test are directly related to the jurisprudence environment. They were lawyers, who had carried out jurisprudential consultations in the last week 19,20,21. The participants were assigned diverse topics such as “Habeas data” or “resignation”. Users navigate in the three search engines, performing the search and applying filters to find more specifically what they require. Once the test was carried out, the participants proceeded to take the questionnaire where the usability and general characteristics of three jurisprudence search engines were compared. They were asked to rate whether they fully agree or disagree with the statements made by the search engines.
First, the navigation and general design of the three jurisprudence search engines were analyzed. Figure 8 shows the results of the 79 participants. It was evaluated if the users totally agree or disagree with the navigation, the ease of finding the search information, and the overall experience with the search engine.
Regarding the statement that "browsing through the search engine was easy and intuitive," 20 users of the judicial branch search engine indicated that they totally disagreed (11) or disagree (9). Only 1 was neutral. Regarding the Relatoria of the Supreme court of Justice, 11 have a disagree o totally disagree with the statement. Finally, with respect to the Lawyer search engine, 15 agreed and 10 fully agreed that browsing through the search engine was easy and intuitive, none of the surveyed indicated to disagree with the statement.
Figure 8 Easy and intuitive navigation for different search engines. Source: self made
Regarding the ease with which users found the information they were looking for. 9 users disagreed with the ease of search engines of the judicial branch, and 5 with that of the Relatoria. 13 agreed with the ease of the search engine of the judicial branch, and 15 with the search engine of the Relatoria. Regarding Lawyer, 30 users agreed and totally agreed with the ease of the search engine and none disagreed with this statement (see figure 9).
Figure 9 - Search experience for different search engines. Source: self made
Figure 10 shows the results regarding the sentence: "the accuracy in general with the search engine was intuitive, easy to use, efficient and correct". It can be seen that (13) users agreed or completely agreed with this statement in the search engine of the judicial branch, (15) they agreed or completely agreed with that statement in the search engine of the rapporteur. Finally, 28 of the participants agree or totally agree that the Lawyer search engine was intuitive and accurate.
Figure 10 Search accuracy. Source: self made
The development of the search engine allows improving judicial tools in Colombia, on the other hand, this type of research is important since the final product is clearly developed for the priorities of the end users, as observed in the DEVA methodology proposal.
The characteristics and elements of the search engine were identified, to this end, benchmarking was carried out to define the general aspects and components of other jurisprudence search engines. This task allowed having a basis for the design of the jurisprudence search engine Lawyer.
It is observed that 79 people who evaluate the system, the vast majority think that lawyer is better in the different evaluations developed both for user experience and for agility and precision in the searches carried out and components developed.
Additionally, a survey was conducted with 79 professionals from the legal environment. Finally, a use test was carried out to validate the characteristics that jurisprudence users prefer. The developed search Lawyer engine meets the functional requirements and its interface is easy to understand. Moreover, Lawyer includes filters to made search more agile. Lawyer offered a better experience compared to other state-ofthe-art search engines, as demonstrated in the different tests.
- Ordoñez Quintero C, Silva A., Ordonez A, Méndez C, Ordoñez H. Sistema de Indexación de Documentos Jurisprudenciales Soportado en Inteligencia Artificial. risti Rev. ibérica Sist. e Tecnol. informação. 2019; 01:1-16.
- Sánchez W. La usabilidad en Ingeniería de Software : definición y características. Ingnovación. Rep. Investig.. 2011; 2:7-21. Publisher Full Text
- Haaksma T, De Jong M, Karreman J.. Users’ Personal Conceptions of Usability and User Experience of Electronic and Software Products. IEEE Trans. Prof. Commun.. 2018; 61(2):116-132. DOI
- Dhandapani S. Integration of User Centered Design and Software Development Process. 2016;1-5. DOI
- Alegre Chang M. G.. Guía del Sistema Nacional de Gestión Ambiental. Minist. del Ambient. del Sist. Nac. Gestión Ambient.. 2016; 01:1-92. Publisher Full Text
- Mamian M, Anacona J, Ordoñez A, Ordoñez C. C.. La pluriculturalidad en el desarrollo de iniciativas de e-gestión centradas en el usuario. RISTI - Rev. Iber. Sist. e Tecnol. Inf.. 2020;1-14.
- Calambás M. A., Ordóñez A., Chacón A., Ordoñez H.. Judicial precedents search supported by natural language processing and clustering. 2015;372-377. DOI
- Ordoñez J. CC., Ordoñez H, Eraso O, De C.-G., Popayán G. T. I., Urbano F.. Jurisprudence search in Colombia based on natural language processing ( NLP ) and Lynked Data. INGE CUC. 2020; 16(2):8-8. Publisher Full Text
- ISO. 9241-210:2010. Ergonomics of human-system interactionpart 210: HUMAN CENTRED DESIGN FOR INTERACTIVE SYSTEMS. Online Browsing Platform(OBP). 2010. Publisher Full Text
- Da Vinci D. L., De Juan D, Huarte D. D., Unidos L. E.. Diseño centrado en el usuario. Diseño centrado en el usuario. 2017; 2(4):52-54.
- Tiganov D, Nguyen Quang Do L, Ali K. Designing UIs for Static Analysis Tools. Queue. 2021; 19(4):97-118. DOI
- Reyes Vera J. M., Berdugo Torres M. I., Machuca Villegas L.. Evaluación de usabilidad de un sistema de administración de cursos basado en la plataforma Lingweb. Ingeniare. Revista chilena de ingeniería. 2016; 24:435-444.
- Pissinati P. de S. C., Évora Y. D. M., Rossaneis M. A., Gvozd R, dos Santos M. S., Haddad M. do C. F. L.. Desenvolvimento de um protótipo de web software de apoio ao planejamento da aposentadoria. Revista Latino-Americana de Enfermagem. 2019; 27
- Canny A, Martinie C, Navarre D, Palanque P, Barboni E, Gris C. Engineering Model-Based Software Testing of WIMP Interactive Applications. Proc. ACM Human-Computer Interact. 2021; 5DOI
- Zorzetti M, Signoretti I, Salerno L, Marczak S, Bastos R. Improving Agile Software Development using User-Centered Design and Lean Startup. Inf. Softw. Technol.. 2022; 141DOI
- Park H. S., Lee G. A., Seo B.-K., Billinghurst M.. User experience design for a smart-mirror-based personalized training system. Multimed. Tools Appl.. 2021; 80(20):31159-31181. DOI
- Perurena Cancio L, Moráguez Bergues M. Usabilidad de los sitios Web, los métodos y las técnicas para la evaluación. Rev. Cuba. Inf. en Ciencias la Salud. 2013; 24(2)Publisher Full Text
- López J, Martínez J. A., Maniega D., Lara P.. RIAs technologies for the development of useroriented systems [Tecnologías RIAs para el desarrollo de sistemas de orientación al usuario]. 2010; 1:224-229. Publisher Full Text
- Heredia Acevedo D, Ceballos Y. F., Sanchez Torres G. Modelo de simulación de eventos discretos para el análisis y mejora del proceso de atención al cliente. Investigación e Innovación en Ingenierías. 2020; 8(2):44-61. Publisher Full Text
- Roccetti M, Prandi C, Mirri S, Salomoni P. Designing human-centric software artifacts with future users: a case study. Human-centric Comput. Inf. Sci.. 2020; 10(1)DOI
- Alnfiai M. A User-centered Design Approach to near Field Communication-based Applications for Children. Int. J. Adv. Comput. Sci. Appl.. 2020; 11(12):486-493. DOI
Descargar CitaEndnote/Zotero/Mendeley (RIS)
- C. Ordoñez Quintero, A. Silva, A. Ordonez, C. Méndez, and H. Ordoñez, “Sistema de Indexación de Documentos Jurisprudenciales Soportado en Inteligencia Artificial,” risti Rev. ibérica Sist. e Tecnol. informação, vol. 01, pp. 1–16, 2019.
- W. Sánchez, “La usabilidad en Ingeniería de Software : definición y características,” Ing-novación. Rep. Investig., no. 2, pp. 7–21, 2011, [Online]. Available: http://www.redicces.org.sv/jspui/bitstream/10972/1937/1/2. La usabilidad en Ingenieria de Software- definicion y caracteristicas.pdf.
- T. Haaksma, M. De Jong, and J. Karreman, “Users’ Personal Conceptions of Usability and User Experience of Electronic and Software Products,” IEEE Trans. Prof. Commun., vol. PP, pp. 1–17, Feb. 2018, doi: 10.1109/TPC.2018.2795398.
- S. Dhandapani, “Integration of User Centered Design and Software Development Process,” in 2016 IEEE 7th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON), 2016, pp. 1–5, doi: 10.1109/IEMCON.2016.7746075.
- Marcos Gabriel Alegre Chang, “Guía del Sistema Nacional de Gestión Ambiental,” Minist. del Ambient. del Sist. Nac. Gestión Ambient., vol. 01, pp. 1–92, 2016, [Online]. Available: file:///C:/Users/SILVANA RUBI/Downloads/guia-snga-minam.pdf.
- M. Mamian, J. Anacona, A. Ordoñez, and C. C. Ordoñez, “La pluriculturalidad en el desarrollo de iniciativas de e-gestión centradas en el usuario,” RISTI - Rev. Iber. Sist. e Tecnol. Inf., no. December, pp. 1–14, 2020.
- M. A. Calambás, A. Ordóñez, A. Chacón, and H. Ordoñez, “Judicial precedents search supported by natural language processing and clustering,” in 2015 10th Computing Colombian Conference (10CCC), 2015, pp. 372–377, doi: 10.1109/ColumbianCC.2015.7333448.
- J. CC.Ordoñez, H. Ordoñez, O. Eraso, C.-G. De, G. T. I. Popayán, and F. Urbano, “Jurisprudence search in Colombia based on natural language processing ( NLP ) and Lynked Data Búsqueda de jurisprudencia en Colombia basada en procesamiento de lenguaje natural ( NLP ) y Lynked Data,” INGE CUC, vol. 16, no. 2, p. 8, 2020, doi: DOI: http://doi.org/10.17981/ingecuc.16.2.2020.22.
- I. 9241-210:2010(EN), “Online Browsing Platform(OBP),” Ergonomics of human-system interaction-part 210: HUMAN CENTRED DESIGN FOR INTERACTIVE SYSTEMS, 2010. https://www.iso.org/obp/ui/iso:std:iso:9241.
- D. L. Da Vinci, D. De Juan, D. D. Huarte, and L. E. Unidos, “Diseño centrado en el usuario,” Diseño centrado en el usuario, vol. 2, no. 4, pp. 52–54, 2017.
- D. Tiganov, L. Nguyen Quang Do, and K. Ali, “Designing UIs for Static Analysis Tools,” Queue, vol. 19, no. 4, pp. 97–118, 2021, doi: 10.1145/3487019.3487026.
- J. M. Reyes Vera, M. I. Berdugo Torres, and L. Machuca Villegas, “Evaluación de usabilidad de un sistema de administración de cursos basado en la plataforma Lingweb ,” Ingeniare. Revista chilena de ingeniería , vol. 24. scielocl , pp. 435–444, 2016.
- P. de S. C. Pissinati, Y. D. M. Évora, M. A. Rossaneis, R. Gvozd, M. S. dos Santos, and M. do C. F. L. Haddad, “Desenvolvimento de um protótipo de web software de apoio ao planejamento da aposentadoria ,” Revista Latino-Americana de Enfermagem , vol. 27. scielo , 2019.
- A. Canny, C. Martinie, D. Navarre, P. Palanque, E. Barboni, and C. Gris, “Engineering Model-Based Software Testing of WIMP Interactive Applications,” Proc. ACM Human-Computer Interact., vol. 5, no. EICS, 2021, doi: 10.1145/3461729.
- M. Zorzetti, I. Signoretti, L. Salerno, S. Marczak, and R. Bastos, “Improving Agile Software Development using User-Centered Design and Lean Startup,” Inf. Softw. Technol., vol. 141, 2022, doi: 10.1016/j.infsof.2021.106718.
- H. S. Park, G. A. Lee, B.-K. Seo, and M. Billinghurst, “User experience design for a smart-mirror-based personalized training system,” Multimed. Tools Appl., vol. 80, no. 20, pp. 31159–31181, 2021, doi: 10.1007/s11042-020-10148-5.
- L. P. C. y Mercedes Moráguez Bergues, “Usabilidad de los sitios Web, los métodos y las técnicas para la evaluación,” Rev. Cuba. Inf. en Ciencias la Salud, vol. 24, no. 2, 2013, [Online]. Available: http://www.rcics.sld.cu/index.php/acimed/article/view/405.
- J. López, J. A. Martínez, D. Maniega, and P. Lara, “RIAs technologies for the development of user-oriented systems [Tecnologías RIAs para el desarrollo de sistemas de orientación al usuario],” in CISCI 2010 - Novena Conferencia Iberoamericana en Sistemas, Cibernetica e Informatica, 7to Simposium Iberoamericano en Educacion, Cibernetica e Informatica, SIECI 2010 - Memorias, 2010, vol. 1, pp. 224–229, [Online]. Available: https://www.scopus.com/inward/record.uri?eid=2-s2.0-84907068359&partnerID=40&md5=85a4cd9e7f3b98e27fa11d8ae5924c3c.
- D. Heredia Acevedo, Y. F. Ceballos, y G. Sanchez Torres, "Modelo de simulación de eventos discretos para el análisis y mejora del proceso de atención al cliente", Investigación e Innovación en Ingenierías, vol. 8, n.º 2, pp. 44-61, 2020. DOI: https://doi.org/10.17081/invinno.8.2.3639
- M. Roccetti, C. Prandi, S. Mirri, and P. Salomoni, “Designing human-centric software artifacts with future users: a case study,” Human-centric Comput. Inf. Sci., vol. 10, no. 1, 2020, doi: 10.1186/s13673-020-0213-6.
- M. Alnfiai, “A User-centered Design Approach to near Field Communication-based Applications for Children,” Int. J. Adv. Comput. Sci. Appl., vol. 11, no. 12, pp. 486–493, 2020, doi: 10.14569/IJACSA.2020.0111259.