Comparative Study of Legal Information Systems: Differences between Systems Using Different Search Strategies
Vol.11,No.22(2020)
Use of legal information retrieval systems is a common part of every lawyer's life. In principle, it is possible to use two approaches to searching: mediated and non-mediated. As a part of the mediated search, we rely on the context provided by the provider of the information system. This might entail assignment of document with particular topic or labelling court decision as related to particular provision. As a part of the non-mediated search, user relies on herself. She formulates full-text queries to search expressions representing the issue at hand.
In this text, I focus on comparison of results provided by legal information systems ASPI, Beck-online and Codexis in response to comparable search queries employing both mediated and non-mediated search.
The introduction and the literature review is followed by part dedicated to research questions. The fourth part described the method used to compare the sys- tems and the data used for the comparison. In the fifth part, I describe the results and the sixth part is devoted to discussion of these results in wider context. The last part presents a conclusion and contains recommendations regarding the selection of search strategies.
In conclusion, use of different systems for mediated search leads to significant differences between systems. On the contrary, when including non-mediated search (full-text queries), results provided by individual systems are more similar to each other, but generally less precise compared to mediated search.
legal information retrieval; search strategies; comparison
s. 219–260
Jakub Harašta
Institute of Law and Technology, Faculty of Law, Masaryk University
[1] ARREDONDO, Pablo. Shepard for a Day: A Novel Class Exercise for Teaching Citators. Legal Reference Service Quarterly, 2015, roč. 34, č. 3, s. 239-244. https://doi.org/10.1080/0270319X.2015.1081009
[2] ASHLEY, Kevin. Artificial Intelligence and Legal Analytics: New Tools for Law Practice in the Digital Age. Cambridge: Cambridge University Press, 2017. https://doi.org/10.1017/9781316761380
[3] BING, Jon. Legal Information Retrieval Systems: The Need for and the Design of Ex- tremely Simple Retrieval Strategies. Computer/Law Journal, 1978, roč. 1, č. 1, s. 379-400.
[4] BING, Jon. Performance of Legal Text Retrieval Systems: The Curse of Boole. Law Library Journal, 1987, roč. 79, č. 2, s. 187-202.
[5] BLAIR, David a M. E. MARON. An Evaluation of Retrieval Effectiveness for a Full-Text Document-Retrieval System. Communications of the ACM, 1985, roč. 28, č. 3, s. 289-299. https://doi.org/10.1145/3166.3197
[6] BOBEK, Michal. Výzkum v právu: reklama na Nike anebo kvantová fyzika? Jurisprudence, 2016, roč. 25, č. 6, s. 3-10.
[7] DABNEY, Daniel. The Curse of Thamus: An Analysis of Full-Text Legal Document Re- trieval, Law Library Journal, 1986, roč. 78, č. 5, s. 5-40.
[8] GERSON, Kevin. Evaluating Legal Information Retrieval Systems: How Do the Ranked-Re- trieval Methods of WESTLAW and LEXIS Measure Up? Legal Reference Services Quarterly, 1999, roč. 17, č. 4, s. 53-67. https://doi.org/10.1300/J113v17n04_04
[9] GRABMAIR, Matthias, Kevin ASHLEY, Ran CHEN, Preethi SURESHKUMAR, Chen WANG, Eric NYBERG a Vern WALKER. Introducing LUIMA: an Experiment in Legal Conceptual Retrieval of Vaccine Injury Decisions Using a UIMA Type System and TOOLS. Proceedings of ICAIL 2015. S. 69-78. https://doi.org/10.1145/2746090.2746096
[10] HAFNER, Carole. Conceptual Organization of Case Law Knowledge Bases. Proceedings of ICAIL 1987. S. 35-42. https://doi.org/10.1145/41735.41740
[11] HARAŠTA, Jakub. Nejednoznačnost odkazů k soudním rozhodnutím a možnosti řešení. Revue pro právo a technologie, roč. 6, č. 11, s. 15-28.
[12] HELLYER, Paul. Evaluating Shepard’s, KeyCite, and BCite for Case Validation Accuracy. Law Library Journal, 2018, roč. 110, č. 4, s. 449-476.
[13] KNAPP, Melanie a Rob WILLEY. Comparison of Research Speed and Accuracy Using Westlaw Classic and WestlawNext. Legal Reference Services Quarterly, 2013, roč. 32, č. 1-2, s. 126-141. https://doi.org/10.1080/0270319X.2013.766487
[14] KOSAŘ, David a Jan PETROV. Jak vybrat „případy“ do případové studie a pracovat s nimi v právu: poznatky z výzkumu na pomezí práva a politologie. Jurisprudence, 2016, roč. 25, č. 6, s. 21-30.
[15] MART, Susan Nevelow. The Case for Curation: The Relevance of Digest and Citator Results in Westlaw and Lexis. Legal Reference Services Quarterly, 2013, roč. 32, č. 1-2, s. 13-53. https://doi.org/10.1080/0270319X.2013.759036
[16] MAXWELL, Tamsin a Burkhard SCHAFER. Concept and Context in Legal Information Retrieval. Proceedings of Jurix 2008. S. 63-72.
[17] SORMUNEN, Eero. Extensions to the STAIRS Study – Empirical Evidence for the Hypo- thesised Ineffectiveness of Boolean Queries in Large Full-Text Databases. Information Retrieval, 2001, roč. 4, č. 3-4, s. 257-273. https://doi.org/10.1023/A:1011950323099
[18] ŠAVELKA, Jaromír, Matěj MYŠKA, Adam PTAŠNIK a Danuše SPÁČILOVÁ. Právní infor- mační systémy. Brno: Tribun EU, 2011.
[19] TAYLOR, William L. Comparing KeyCite and Shepard’s for Completeness, Currency, and Accuracy. Law Library Journal, 2000, roč. 92, č. 2, s. 127-142. https://doi.org/10.1023/A:1010043220792
[20] TURTLE, Howard. Natural Language vs. Boolean Query Evaluation: A Comparison of Retrieval Performance. Proceedings of SIGIR 1994. S. 212-220. https://doi.org/10.1007/978-1-4471-2099-5_22
[21] URBÁNIKOVÁ, Marína a Hubert SMEKAL. Právní věda a právní psaní: postačí vždy jako výzkumná metoda "číst, přemýšlet a psát"? Jurisprudence, 2017, roč. 26, č. 4, s. 37-41.
[22] VAN OPIJNEN, Marc a Cristiana SANTOS. On the Concept of Relevance in Legal Infor- mation Retrieval. Artificial Intelligence and Law, 2017, roč. 25, č. 1, s. 65-87. https://doi.org/10.1007/s10506-017-9195-8
[23] WILENSKY, Beth. When Should We Teach Our Students to Pay Attention to the Costs of Legal Research? Perspectives: Teaching Legal Research and Writing. 2016, roč. 26, č.1-2, s. 41-46.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Copyright © 2020 Jakub Harašta