You are here

Enhancing the Scaling and Accuracy of Text Analytics Using Joint Inference


OBJECTIVE: To conduct basic research and define and evaluate algorithms which perform joint inference for the purpose of enhancing the performance of deep Natural Language Understanding (NLU). DESCRIPTION: A key goal of deep Natural Language Understanding (NLU) is to understand the sophisticated relationships that can be expressed in text both explicitly through explicit phrases like"John Married Mary in 2002", but also through implicit information that we gain from performing inference across several relationships, e.g.,"Mary was divorced in 2007 and goes by her maiden name Mary Smith". From this information, we discover a non-obvious co-reference and that John (referenced above) is also divorced. To accomplish this task, successful NLU systems need to extract multiple relations (e.g., both marriage and divorce) and possess relevant domain knowledge (e.g., the fact that divorce ends marriage and is symmetric). Recent years have seen an explosion of successes in combining probability and (subsets of) first-order logic respectively. First order logic handles complexity and probability handles uncertainty. Traditional AI has inevitably overtaxed resources when scaling up the environment. Inference has been the biggest bottleneck for the use of Statistical Relational Learning (SRL) Models in practice. Inference in SRL Models is very hard. These properties have led to a surge in interest in lifted probabilistic inference algorithms that exploit redundancies to speed-up inference, ultimately avoiding explicit state enumeration by manipulating first-order state representations directly. Therefore, inference becomes a crucial issue due to learning becoming harder and the greater complexity for users. Lifted inference, not grounded, in first order logic is often faster, more compact and can provide more structure for optimization. Lifted inference deals with groups of random variables at a first order level and possesses the ability to carry out probabilistic inference in a relational probabilistic model without needing to reason about each individual separately. Lifted inference has the potential to exploit interchangeability in a domain, queries can be returned without instantiating all the objects in the domain and it exploits shared correlations and symmetries in the data and model, while receiving and sending the same message. The use of Joint inference to improve deep Natural Language Understanding can form the basis to represent a large variety of linguistic information about articles in one logic-based, formal notation. Methods, like probabilistic logic, can be applied for learning and reasoning. Research needs to be performed to evaluate the various lifted inference methodologies and techniques and evaluate not just how well the performance but against what requirements. Lifted inference contains both graph based and search based criteria. The graph-based implementations include exact, approximate and pre-processing subgroups. The approximate subgroup can be further quantified as deterministic approximation, sampling and interval methods. Examples of lifted exact inference are lifted variable elimination (VE), lifted VE plus aggregation, Bisimulated VE and First Order VE (FOVE). Example of Lifted Pre-processing Inference is Fast Reduction of Grounded Markov Logic Networks (MLNs) (FROG). Example Lifted interval Inference is Anytime Belief Propagation (BP). Examples of Lifted sampling inference are MCMC techniques, logic particle filter and lazy inference. PHASE I: Study: (1) Scalability of the different lifted inference techniques (2) Trade-off between use of background knowledge to improve the final results versus the effort required to encode such knowledge (3) Performance versus different kinds of problems (4) Evaluate the performance and scalability of the different techniques for joint inference at the cross-document or corpus level PHASE II: Leverage the results from phase I to develop a framework to perform text analytics over an information repository using the various reasoning and learning approaches to joint inference. Demonstrate the capabilities on an end-to-end prototype to perform tasks like answering specific queries without considering the entire model and/or the entire evidence. Demonstrate joint inference over cross-document and multiple knowledge bases. Explore combining lifted inference with dual decomposition. PHASE III: The algorithms for Lifted inference have a variety of applications. They include social networks, object recognition, link prediction, activity recognition, model counting, bio-medical applications and relational reasoning and learning. Fundamental building block to improve current capabilities. REFERENCES: 1. Tutorial on Lifted Inference in Probabilistic Logical Models, Twenty-second international joint conference on Artificial Intelligence (IJCAI) 2011. Kersting, Natarajan and Poole. 2. Efficient Inference Methods for Probabilistic Logical Models, Sriraam Natarajan, University of Wisconsin-Madison, at 3. Markov Logic: An Interface Layer for Artificial Intelligence By Pedro Domingos, Daniel Lowd, Morgan & Claypool Publishers, Aug 15, 2009
US Flag An Official Website of the United States Government