You are here
Knowledge-aided Interface for Big Data Streams
Phone: (321) 591-7371
Phone: (919) 244-3946
Big data challenges across Department of Defense (DOD) domains are increasingly problematic for tactical level decision making. Data collections in the open source and military channels are growing at such a staggering rate that it exceeds our ability to store and manage, perform computation and analysis, and maintain data security [1, 2]. Indeed, one author refers to the inability to handle big data as the new helplessness age , a reaction to the inability of information processing algorithms to rapidly extract key elements of information to aid decision making in time constrained environments. Architectural limitations are a major constraint to discovering knowledge from big data stores that represent complex combinations of many data types . Due to the exponential increase in data, combined with the limitations in processing capability, it is unlikely that Warfighters operating in uncertain and unfamiliar cultural environments will benefit from knowledge discovery capabilities any time soon. To reduce vulnerability and risk for Warfighters from unknown threats, new and innovative approaches are needed for data collection, processing, and user interface designs. Promising approaches in this space include data streams , interactive exploration and hypothesis testing of data , and temporal segmentation of large text corpora . Addressing data stream computation is recognition that tactical decisions require a very small subset of all data available in military databases and that valuable data may often be separate from the traditional hard sciences approach to persistent collection and quantitative analysis . In data stream processing, data arrives in continuous, high-volume, fast and time-varying streams . Clustering, classification, and association algorithms may be useful for mining data streams, but transferring results over a wireless network with limited bandwidth could prove challenging for tactical units . Interactive exploration and hypothesis testing of data streams could serve to filter large amounts of information for specific tactical knowledge requirements. Frequently, Warfighters dont know the right questions to ask and have very limited opportunities to explore options for potential outcomes. Bio-inspired applications for interface design and collaboration in a visual domain could improve interface designs. Biological features that might be adapted to interfaces could include autonomy, scalability, adaptability, and robustness , each designed to detect data patterns, identify anomalies, and extract knowledge from enormous volumes of data. The key component of achieving success in this particular problem area is to ensure that computer and social scientists work closely together  in order to develop sufficiently robust algorithms with greater reliance on reasoning that allow a domain-relevant interpretation of actionable patterns of behavior and meaning for informed decision making . Temporal segmentation of large text corpora may provide a method by which data may be filtered at tactical levels for rapid processing and knowledge extraction. Using text open sources (e.g., newspapers, blogs, Tweets, Facebook posts) would provide Warfighters with near-real time insight into semantic tones of localized text . The potential value of this approach would be to allow users to infer a timeline of factors correlated with ideas identified from analysis of public discussion in text corpora. The challenges with this topic are storage and management of big data, which may contribute to an inability to validate and qualify each data item. Also, careful design of systems is necessary to match user needs and the technologies used for analytics and visual display of information. In addition, accessing very large quantities of semi- or unstructured data is problematic and limited by available storage applications and hardware. Finally, user needs must be supported by computational processes, with these expressed in ways that are consistent with the larger social system in which the user operates. Frequently, user studies concentrate on the micro-system of the individual user and fail to consider the wider range of opportunities, challenges, and constraints. The current topic seeks to address those challenges by focusing on a new and innovative data collection/storage/processing method that can reduce noise in large data while keeping relevant data streams for processing. It also will explore interactive user interface designs that allow temporal segmentation, or other useful algorithms, which should consider bio-inspired applications. Finally, placing the user within the larger social system for developing filtering and visual methods will provide a unitary perspective for knowledge discovery and dissemination.
* Information listed above is at the time of submission. *