Language Resources and Evaluation Conference 2012 (LREC 2012)

27 May, 2012
Istanbul, Turkey

Context:

The legal domain represents a primary candidate for web-based information distribution, exchange and management, as testified by the numerous e-government, e-justice and e-democracy initiatives worldwide. The last few years have seen a growing body of research and practice in the field of Artificial Intelligence and Law which addresses a range of topics: automated legal reasoning and argumentation, semantic and cross-language legal information retrieval, document classification, legal drafting, legal knowledge discovery and extraction, as well as the construction of legal ontologies and their application to the law domain. In this context, it is of paramount importance to use Natural Language Processing techniques and tools that automate and facilitate the process of knowledge extraction from legal texts.

Since 2008, the SPLeT workshops have been a venue where researchers from the Computational Linguistics and Artificial Intelligence and Law communities meet, exchange information, compare perspectives, and share experiences and concerns on the topic of legal knowledge extraction and management, with particular emphasis on the semantic processing of legal texts. Within the Artificial Intelligence and Law community, there have also been a number of dedicated workshops and tutorials specifically focussing on different aspects of semantic processing of legal texts at conferences such as JURIX-2008, ICAIL-2009, ICAIL-2011, as well as in the International Summer School “Managing Legal Resources in the Semantic Web” (2007, 2008, 2009, 2010, 2011).

To continue this momentum and to advance research, a 4th Workshop on “Semantic Processing of Legal Texts” is being organized at the LREC-2012 conference to bring to the attention of the broader LR/HLT (Language Resources/Human Language Technology) community the specific technical challenges posed by the semantic processing of legal texts and also share with the community the motivations and objectives which make it of interest to researchers in legal informatics. The outcome of these interactions are expected to advance research and applications and foster interdisciplinary collaboration within the legal domain.

New to this edition of the workshop are two sub-events (described below) to provide common and consistent task definitions, datasets, and evaluation for legal-IE systems along with a forum for the presentation of varying but focused efforts on their development.

The main goals of the workshop and associated events are to provide an overview of the state-of-the-art in legal knowledge extraction and management, to explore new research and development directions and emerging trends, and to exchange information regarding legal language resources and human language technologies and their applications.

Sub-events:

Dependency Parsing
The first sub-event will be a shared task specifically focusing on dependency parsing of legal texts: although this is not a domain-specific task, it is a task which creates the prerequisites for advanced IE applications operating on legal texts, which can benefit from reliable preprocessing tools. For this year our aim is to create the prerequisites for more advanced domain-specific tasks (e.g. event extraction) to be organized in future SPLeT editions. We strongly believe that this could be a way to attract the attention of the LR/HLT community to the specific challenges posed by the analysis of this type of texts and to have a clearer idea of the current state of the art. The languages dealt with will be Italian and English. A specific Call for Participation for the shared task is available in a dedicated page.

Semantic Annotation
The second sub-event will be an online, manual, collaborative, semantic annotation exercise, the results of which will be presented and discussed at the workshop. The goals of the exercise are: (1) to gain insight on and work towards the creation of a gold standard corpus of legal documents in a cohesive domain; and (2) to test the feasibility of the exercise and to get feedback on its annotation structure and workflow. The corpus to be annotated will be a selection of documents drawn from EU and US legislation, regulation, and case law in a particular domain (e.g. consumer or environmental protection). For this exercise, the language will be English. A specific Call for Participation for this annotation exercise is available in a dedicated page.

Areas of Interest:

The workshop will focus on the topics of the automatic extraction of information from legal texts and the structural organisation of the extracted knowledge. Particular emphasis will be given to the crucial role of language resources and human language technologies.

Papers are invited on, but not limited to, the following topics:

• Construction, extension, merging, customization of legal language resources, e.g. terminologies, thesauri, ontologies, corpora
• Information retrieval and extraction from legal texts
• Semantic annotation of legal text
• Legal text processing
• Multilingual aspects of legal text semantic processing
• Legal thesauri mapping
• Automatic Classification of legal documents
• Logical analysis of legal language
• Automated parsing and translation of natural language legal arguments into a logical formalism
• Dialogue protocols for legal information processing
• Controlled language systems for law

Workshop Schedule – TBA:

Workshop Registration and Location – TBA:

Webpage URL:

http://wyner.info/LanguageLogicLawSoftware/?p=1233

Important Dates:

• Submission: 10 February 2012
• Acceptance Notification: 5 March 2012
• Final Version: 23 March 2012
• Workshop date: 27 May 2012

Author Guidelines:

Submissions are solicited from researchers working on all aspects of semantic processing of legal texts. Authors are invited to submit papers describing original completed work, work in progress, interesting problems, case studies or research trends related to one or more of the topics of interest listed above. The final version of the accepted papers will be published in the Workshop Proceedings.

Short or full papers can be submitted. Short papers are expected to present new ideas or new visions that may influence the direction of future research, yet they may be less mature than full papers. While an exhaustive evaluation of the proposed ideas is not necessary, insight and in-depth understanding of the issues is expected. Full papers should be more well developed and evaluated. Short papers will be reviewed the same way as full papers by the Program Committee and will be published in the Workshop Proceedings.

Full paper submissions should not exceed 10 pages, short papers 6 pages; both should be typeset using a font size of 11 points. Style files will be made available by LREC for the camera-ready versions of accepted papers. Papers should be submitted electronically, no later than February 10, 2012. The only accepted format for submitted papers is Adobe PDF.

Submit papers to:

Submission will be electronic using START paper submission software available at:

https://www.softconf.com/lrec2012/SPLeT2012/

Note that when submitting a paper through the START page, authors will be asked to provide essential information about resources (in a broad sense, i.e. also technologies, standards, evaluation kits, etc.) that have been used for the work described in the paper or are a new result of your research. For further information on this new initiative, please refer to:

http://www.lrec-conf.org/lrec2012/?LRE-Map-2012

Publication:

After the workshop a number of selected, revised, peer-reviewed articles will be published in a Special Issue on Semantic Processing of Legal Texts of the AI and Law Journal (Springer).

Contact Information:

Address any queries regarding the workshop to:

lrec_legalWS@ilc.cnr.it

Program Committee Co-Chairs:

Enrico Francesconi (National Research Center, Italy)
Simonetta Montemagni (National Research Center, Italy)
Wim Peters (University of Sheffield, UK)
Adam Wyner (University of Liverpool, UK)

Program Committee (Preliminary):

Kevin Ashley (University of Pittsburgh, USA)
Johan Bos (University of Rome, Italy)
Daniele Bourcier (Humboldt Universitat, Germany)
Pompeu Casanovas (Universitat Autonoma de Barcelona, Spain)
Jack Conrad (Thomson Reuters, USA)
Matthias Grabmair (University of Pittsburgh, USA)
Antonio Lazari (Scuola Superiore S.Anna, Italy)
Leonardo Lesmo (Universita di Torino, Italy)
Marie-Francine Moens (Katholieke Universiteit Leuven, Belgium)
Thorne McCarty (Rutgers University, USA)
Raquel Mochales Palau (Catholic University of Leuven, Belgium)
Paulo Quaresma (Universidade de Evora, Portugal)
Tony Russell-Rose (UXLabs, UK)
Erich Schweighofer (Universitat Wien, Austria)
Rolf Schwitter (Macquarie University, Australia)
Manfred Stede (University of Potsdam, Germany)
Daniela Tiscornia (National Research Council, Italy)
Tom van Engers (University of Amsterdam, Netherlands)
Giulia Venturi (Scuola Superiore S.Anna, Italy)
Vern R. Walker (Hofstra University, USA)
Radboud Winkels (University of Amsterdam, Netherlands)

On Rule Extraction from Regulations
Adam Wyner and Wim Peters

Abstract
Rules in regulations such as found in the US Federal Code of Regulations can be expressed using conditional and deontic rules. Identifying and extracting such rules from the language of the source material would be useful for automating rulebook management and translating into an executable logic. The paper presents a linguistically-oriented, rule-based approach, which is in contrast to a machine learning approach. It outlines use cases, discusses the source materials, reviews the methodology, then provides initial results and future steps.

Populating an Online Consultation Tool
Sarah Pulfrey-Taylor, Emily Henthorn, Katie Atkinson, Adam Wyner, and Trevor Bench-Capon

Abstract
The paper addresses the extraction, formalisation, and presentation of public policy arguments. Arguments are extracted from documents that comment on public policy proposals. Formalising the information from the arguments enables the construction of models and systematic analysis of the arguments. In addition, the arguments are represented in a form suitable for presentation in an online consultation tool. Thus, the forms in the consultation correlate with the formalisation and can be evaluated accordingly. The stages of the process are outlined with reference to a working example.

Shortlink to this page.

By Adam Wyner
Distributed under the Creative Commons
Attribution-Non-Commercial-Share Alike 2.0

At the links below, you can find the slides and hands on materials on GATE for the LEX summer school on Managing Legal Resources in the Semantic Web.

GATE Legislative Rulebook

By Adam Wyner
Distributed under the Creative Commons
Attribution-Non-Commercial-Share Alike 2.0

Towards Annotating and Extracting Textual Legal Case Elements
Adam Wyner

Abstract
The paper presents an outline of a method for semantic, conceptual search in legal case documents using the GATE tool.

By Adam Wyner
Distributed under the Creative Commons
Attribution-Non-Commercial-Share Alike 2.0

As this was my second time at the GATE summer school, I sat in on the Advanced GATE session. All the slides and all the materials for hands on exercises are available on the GATE Summer School Wiki. In my week, we covered the following:

• Module 9: Ontologies and Semantic Annotation
  • Introduction to Ontologies
  • GATE Ontology Editor
  • GATE Ontology Annotation Tools for Entities and Relations
  • Automatic Semantic Annotation in GATE
  • Measuring Performance
  • Using the Large Knowledge Base gazetteer (LKB)

• Module 10: Advanced GATE Applications
  • Customising ANNIE
  • Working with different languages
  • Complex applications
  • Conditional Processing
  • Section-by-section processing

• Module 11: Machine Learning
  • Machine learning and evaluation concepts
  • Using ML in GATE
  • Engines and algorithms)
  • Entity learning hands-onl session
  • Relation extraction hands-on session

• Module 12: Opinion Mining
  • Introduction to opinion mining and sentiment analysis
  • Using GATE tools to perform sentiment analysis
  • Machine learning for sentiment analysis hands-on session
  • Future directions for opinion mining

• Module 13: Semantic Technology and Linked Open Data: Basics, Tools, and Applications
  • Linked Open Data: Introduction of key principles and some key tools (FactForge, LinkedLifeData)
  • Semantic Annotation with Linked Data
  • Semantic Search

Textual Information Extraction from Legal Resources using GATE

The slides of all the talks, including mine, are available:

ISKO-UK Legal Know-How meeting

In a couple of weeks, ISKO will also add mp3s of the talks, so one can see the slides and hear the talks. Nice way to do things, as remarks and narration are almost more crucial than the slides themselves.

By Adam Wyner
Distributed under the Creative Commons
Attribution-Non-Commercial-Share Alike 2.0

Lexical Semantics and Expert Legal Knowledge towards the Identification of Legal Case Factors

accepted for presentation at JURIX 2010. The list of accepted papers is here. The paper will appear in the proceedings, but it is available by clicking on the paper title above.

Abstract
Legal case factors are textually represented facts which are represented in reported legal case decisions. Precedent decisions contribute to the decision of a case under consideration. As textually represented facts, factors linguistically encode semantic properties and relationships among the entities which can be leveraged to identify and extract the legal case factors from decisions. We integrate legal and linguistic resources in a text analysis tool with which we annotate textual passages. Using annotations tailored to legal case factors, the legal researcher can rapidly zero in on textual spans which represent specific combinations of factors, participants, and semantic properties which bear on who played what role with respect to a factor. The research reports progress on the development of a tool.

By Adam Wyner
Distributed under the Creative Commons
Attribution-Non-Commercial-Share Alike 2.0

Features of cases

Legal cases contain a wealth of detailed information such as:

• Case citation.
• Names of parties.
• Roles of parties, meaning plaintiff or defendant.
• Sort of court.
• Names of judges.
• Names of attorneys.
• Roles of attorneys, meaning the side they represent.
• Final decision.
• Cases cited.
• Relation of precedents to current case.
• Case structural features such as sections.
• Nature of the case, meaning using keywords to classify the case in terms of subject (e.g. criminal assault, intellectual property, ….)

With respect to these features, one would want to make a range of queries (using some appropriate query language).

• In what cases has company X been a defendant?
• In what cases has attorney Y worked for company X, where X was a defendant?
• What are the final decisions for judge Z?
• If the case concerns criminal assault, was a weapon used?

We initially based our work on Bransford-Koons Ph.D. Thesis 2005, commenting on, adapting, and adding to it. We used cases from California Criminal Courts which were used in that work since the lists and rules are highly specific.

Output

We have the following sample outputs from our lists and rules applied to People v. Coleman, 117 Cal App. 2d 565. In the first figure, we find the address, court district, citation, case name, counsels for each side, and the roles. There are aspects which need to be further cleaned up, but this gives a flavour of the annotations.

In the second figure, we focus on additional information such as structural sections (e.g. Opinion), the name of the judge, and terms having a bearing on criminal assault and weapons.

In the final figure, we identify the decision.

GATE

In the archive, we have the application, lists, JAPE rules, and graphics. The lists.def file in this archive are associated with the various other lists. The JAPE rules may have different names from what is found in the application and discussed below, but (so far as we understand), this should make no difference in the functionality.

Lists

Gazetteer lists which were used are the following; these are lists contained in a master list labelled DSAGaz. We samples and comment below.

• lists.def. The gazetteer list which contains the lists below. When importing this along with the standard ANNIE list, this list is renamed in the application.
• attack_words.lst. Actions that can be construed as attacks such as hit, hitting, throw, thrown, threw,….
• intention.lst. Terms for intention such as intend, intends, intending,…, expect, expects,….
• judgements.lst. Terms related to judgment such as granted, denied, reversed, overturned, remanded,….
• judgeindicator.lst. The indicator J.. This is a problematic indicator if it is part of an individual’s name.
• criminal_assault.lst. Terms related to assault such as assault, violent injury, ability,…. It is unclear just how cohesive this set of terms is.
• legal_appellate_districts.lst. A list of appellate districts such as Fifth Appellate District, Fifth Dist.,….
• legal_casenames.lst. Terms that can be used to indicate case names such as v., In Re,
• legal_counselnames.lst. Terms for counselor titles such as Attorney General, Deputy Public Defender,….
• legal_general.lst. Terms for footnotes or numbering sections such as fn., No.,….
• legal_opinion_sections.lst. Terms for sections of legal opinion such as concurring, counsel, dissenting, opinion,….
• legal_coa.lst. Terms for causes of action such as aggravated assault, assault, breaking and entering, burglary, robbery,….
• legal_code_citations.lst. Code citation information such as Civ. Code, Penal Code,….
• us_district_abb_01.lst. Abbreviations for legal districts such as Cal., P., Wis.,….
• us_context_abb_01.lst. Abbreviations for participant roles such as App., Rptr,….
• legal_citations.lst. Abbreviations for citations and related to districts such as Cal.2d, Cal.App. 3d,….
• legal_parties.lst. Terms for legal roles such as amicus curie, appellant, appellee, counsel, defendant, plaintiff, victim, witness,….
• lower_courts.lst. Phrases for other courts such as Municipal Court of, Superior Court of,….
• possible_weapons.lst. A list of items that could be weapons such as automobile, bat, belt,….
• weapons.lst. A list of items that are weapons such as assault rifle, axe, club, fist, gun,….

Discussion of Lists

We used some of the lists directly from Bransford-Koons 2005, but they are clearly in need of reconstruction and extension. A general problem is that the lists are defined for US case law and particularly the California district courts. Thus, we cannot simply apply the lists to different jurisdictions, e.g. the United Kingdom; the lists and rules must be relativised to different contexts. More technically, lists have alternative graphical (capital or lower case) or morphological forms, which would be better addressed using a Flexible Gazetteer. In addition, it is unclear how one could bound the range of relevant terms appropriately and give them interpretations that are relevant to the context; in general, a lexicon or ontology could give us a better list of terms, but we must find some means to construe them as need be in the legal context. For example, we have a range of attack action terms such as hit, hitting, throw, thrown, threw,….; in some contexts these actions need not be construed as attack, e.g. baseball. Some means needs to be found to ascribe the appropriate interpretation in context. A related issue is whether we must list all alternative forms of some terms (also taking into consideration spaces) or whether we can better write JAPE rules; this is relevant for the list of appellate districts, where we find both abbreviations and alternative elements of information as in Fifth Appellate District, Fifth Appellate District Div 1, and Fifth Appellate District, Division 1. Along these lines, we would prefer a systematic means to relate abbreviations to the terms they abbreviate. In our view, more general solutions are better than specific ones which list information; lists ought to be contain arbitrary information, while JAPE rules construct systematic information.

JAPE Rules

Given the lists, we have JAPE rules to annotate the relevant portions of text.

• AppellantCounsel: annotates the appellant counsel.
• RespondentCounsel: annotates the respondent counsel.
• DSACounsellor: annotates counsels.
• SectionsTerm: annotates sections relative to the list of section terms.
• CaseRoleGeneral
• DSACaseName2: annotates the case name.
• DSACaseName: annotates the case name.
• DSACaseCit: annotates the case citation.
• CriminalAssault: annotates terms for criminal assault.
• CauseOfAction: annotates for causes of action.
• AttackTerm: annotates attack terms.
• AppellateDistrict: annotates districts of courts.
• DecisionStatement: annotates a sentence as the decision statement.
• JudgementTerm: annotates terms related to judgement.
• JudgeName: annotates the names of judges.
• JudgeInd: annotates the judge name indicator.
• IntentTerm: annotates terms of intent.

Discussion

Some of these rules annotate sentences, while others annotate entities with respect to some property. Some of the rules don’t work quite as well as we would wish and could stand further refinement such as the rule for the roles of counsels; the solution we have is rather ad hoc. Nonetheless, as a first pass, the lists and rules give some indication of what is possible.

Order of application

• Document Reset PR
• RegexSentenceSplitter
• ANNIE English Tokeniser
• ANNIE POS Tagger
• MorphologicalAnalyzer
• DSAGaz
• AnnieGaz
• Flexible Gazetteer
• NPChunker
• ANNIE NE Transducer
• IntentTerm
• JudgeInd
• JudgeName
• JudgementTerm
• DecisionStatement
• Weapons
• AppellateDistrict
• AttackTerm
• CauseOfAction
• CriminalAssault
• DSACaseCit
• DSACaseName
• DSACaseName2
• DSACaseNameAZW
• CaseRoleGeneral
• SectionsTerm
• DSACounsellor
• RespondentCounsel
• AppellantCounsel

Discussion

Despite the limitations, this gives some useful, preliminary results which can easily be built upon. Moreover, we know of no other public, open system of annotating case elements (or factors).

By Adam Wyner
Distributed under the Creative Commons
Attribution-Non-Commercial-Share Alike 2.0

Introduction to ANNIC

ANNIC is an annotation indexing and retrieval system. It is integrated with the data stores, where results of annotations on a corpus can be saved. Once a processing pipeline is run over the corpus, we can use ANNIC to query and inspect the contexts where annotations appear; the queries are in a subset of the JAPE language, so can be complex. The results of the queries are presented graphically, making them easy to understand. As such, ANNIC is a very useful tool in the development of rules as one can discover and test patterns in corpora. There is also an export facility, so the results can be presented in a file, but this is not a full information extraction system such as one might want with templates.

For later, but important to know from the documentation: “Be warned that only the annotation sets, types and features initially indexed will be updated when adding/removing documents to the datastore. This means, for example, that if you add a new annotation type in one of the indexed document, it will not appear in the results when searching for it.” This implies that where one adds new annotations to the pipeline, one should delete the old data store and create another one with respect to the new results. For example, if one ran the pipeline without POS, one cannot add POS later and inspect it in the pipeline.}

Further details on ANNIC are available at GATE documentation on ANNIC and there is an online video.

Instantiating the serial data store

The following steps are used to create the requisite parts and inspect them with ANNIC. One starts with an empty GATE, then adds processing resources, language resources, and pipelines since these can all be related to the data store in a later step. This material is adapted or adopted from the GATE ANNIC documentation, cutting out many of the options. To instantiate a serial data store (SSD), which is how the annotated documents are saved and searched. The application, lists, and rules that this example uses is from Information Extraction of Legal Case Factors.

• RC on Datastores > Create datastore.
• From the drop-down list select “Lucene Based Searchable DataStore”.
• At the input window, provide the following parameters:
• DataStore URL: Select an empty folder where the data store is created.
• Index Location: Select an empty folder. This is where the index will be created.
• Annotation Sets: Provide the annotation sets that you wish to include or exclude from being indexed. There are options here, but we want to index all the annotation sets in all the documents, so make this list empty.
• Base-Token Type: These are the basic tokens of any document (e.g. Token) which your documents must in order to get indexed.
• Index Unit Type: This specifies the unit of Index (e.g. Sentence). In other words, annotations lying within the boundaries of the annotations are indexed (e.g. in the case of Sentence, no annotations that are spanned across the boundaries of two sentences are considered for indexing). We use the Sentence unit.
• Features: Users can specify the annotation types and features that should be included or excluded from being indexed (e.g. exclude SpaceToken, Split, or Person.matches).
• Click OK. If all parameters are OK, a new empty searchable SSD will be created.
• Create an empty corpus and save it to the SSD.
• Populate the corpus with some documents. Each document in the corpus is automatically indexed and saved to the data store.
• Load some processing resources and then a pipeline. Run the pipeline over the corpus.
• Once the pipeline has finished (and there are no errors), save the corpus in the SSD by right clicking on the corpus, then “Save to its datastore”.
• Double click on the SSD file under Datastores. Click on the “Lucene DataStore Searcher” tab to activate the search GUI.
• Now you are ready to specify a search query of your annotated documents in the SSD.

Output

The GUI opens with parts as shown in the following two figures:

Working with the GUI

The figures above show three main sections. In the top section, left section, there is a blank text area in which one can write a query (more on this below); the search query returns the “content” of the annotations. There are options to select a corpus, annotation set, the number of results, the size of the context (e.g. the number of tokens to the left and right of what one searches for). In the central section, one can see a visualisation of annotations and values given the search query. In the bottom section, one has a list of the matches to the query across the corpus, giving the left and right contexts relative to the search results. An annotation rows manager lets one add (green plus sign) or remove (red minus sign) annotation types and features to display in the central section. The bottom section contains the results table of the query, i.e. the text that matches the query with their left and right contexts. The bottom section also contains tabbed panes of statistics such as how many instances of particular annotation appear.

Queries

The queries written in the blank text area are a subset of the JAPE patterns and use the annotations used in the pipeline. Queries are activated by hitting ENTER (or the Search icon). The following are some template patterns that can be used Below we give a few examples of JAPE pattern clauses which can be used as SSD queries.

• String
• {AnnotationType}
• {AnnotationType == String}
• {AnnotationType.feature == feature value}
• {AnnotationType1, AnnotationType2.feature == featureValue}
• {AnnotationType1.feature == featureValue,
  AnnotationType2.feature == featureValue}

Specific queries are:

• Trandes — returns all occurrences of the string where it appears in the corpus.
• {Person} — returns annotations of type Person.
• {Token.string == “Microsoft”} — returns all occurrences of “Microsoft”.
• {Person}({Token})*2{Organization} — returns Person followed by zero or up to two tokens followed by Organization.
• {Token.orth==”upperInitial”, Organization} — returns Token with feature orth with value set to “upperInitial” and which is also annotated as Organization.
• {Token.string==”Trandes”}{Token})*10{Secret} — returns string “Trandes” followed by zero to ten tokens followed by Secret.
• {Token.string ==”not”}({Token})*4{Secret} — returns the string “not”, followed by 4 or less tokens, followed by something annotated with Secret.

An example of a result for the last query is:

Trandes averred nothing more than that it possessed secret.

In ANNIC, the result of the query appears as:

One can write queries using the JAPE operators: | (OR operator), +, and *. ({A})+n means one and up to n occurrences of annotation {A}, and ({A})*n means zero or up to n occurrences of annotation {A}.

Summary

ANNIC is particularly useful in writing and refining one’s JAPE rules. Finally, one’s results can be exported at HTML files.

By Adam Wyner
Distributed under the Creative Commons
Attribution-Non-Commercial-Share Alike 2.0