Focus

Text and multimedia semantic analysis is concerned with the extraction of meaning from documents, images, video, and other multimedia sources. Research in the LSU Semantic Analysis lab focuses on the development of new techniques in semantic analysis, as well as application of techniques to applications such as task automation, information extraction, text and multimedia analytics, social network analysis, intelligence gathering, human computer and human machine interfaces, and cognitive ergonomics.Research in the Semantic Analysis lab covers the automated determination of semantic content (a.k.a. meaning) from multimedia (text, speech, sensors, metadata, images, etc.), and applications to human-computer and human-machine interfaces, task automation, and cognitive ergonomics.

Students pursuing research in semantic analysis generally do so through the PhD and MS programs in Engineering Science with concentration in Information Technology and Engineering (ITE), or through the MS Industrial Engineering program with focus in IT. 

Current Research

  • Multimedia image semantics
  • Affect detection
  • Anaphora resolution
  • Discourse structure identification
  • Ontology / knowledge extraction.
  • Social network mining
  • Natural language processing applications to healthcare

Corpus

The annotated Affect Corpus 2.0 is available for research purposes and can be downloaded from the Corpus tab on the menu.

Data from AVEC 2011 can be downloaded here: Link  

Recent Publications & Presentations

Calix, R. A., Javadpour, L., and Knapp, G. M., "Detection of Affective States from Text and Speech for Real-time Human Computer Interaction", Human Factors: The Journal of the Human Factors and Ergonomics Society, Vol 54 no. 4 (2013), pp. 530-545, doi: 10.1177/0018720811425922.

 Calix, R. A, and Knapp, G. M., "Actor Level Emotion Magnitude Prediction in Text and Speech", Multimedia Tools and Applications, Vol 62, Issue 2 (2013), pp. 319-332, http://dx.doi.org/10.1007/s11042-011-0909-8.

 Calix, R., and Knapp, G.M., "Automatic Detection of Nominal Entities in Speech for Enriched Content Search", Proceedings of the Twenty-Sixth International Florida Artificial Intelligence Research Society Conference (FLAIRS 26), May 2013, pp. 190-195.

 Patwardhan, A, and Knapp, G.M., "Multimodal Affect Analysis for Product Feedback Assessment", Proceedings of the 2013 Industrial and Systems Engineering Research Conference (ISERC), May 2013, pp. 178-186.

Guidry, J and Knapp, G.M., "Discourse Structure Identification for Knowledge Extraction", Proceedings of the 2013 Industrial and Systems Engineering Research Conference (ISERC), May 2013, pp. 214-223.

Halder, A., and Knapp, G.M., "Automatic Extraction of Ontology from Technical Journals", Proceedings of the 2013 Industrial and Systems Engineering Research Conference (ISERC), May 2013, pp. 224-230.

Weldu, Y., and Knapp, G.M., "Automating Construction Activity Generation & Visualization", Proceedings of the 2013 Industrial and Systems Engineering Research Conference (ISERC), May 2013, pp. 241-249.

Khazaeli, M., Javadpour, L., and Knapp, G.M., "Kinect Applications in Construction: From Tracking to Reconstruction", Proceedings of the 2013 Industrial and Systems Engineering Research Conference (ISERC), May 2013, pp. 259-267.

Javadpour, L., & Knapp, G.M., "Evidence Fusion in Coreference Resolution", published abstract & presentation, 2013 Industrial and Systems Engineering Research Conference (ISERC), May 2013.

 Khazaeli, M, Javadpour, L, and Knapp, G.M., "Semantic Scene Understanding", published abstract & presentation, 2013 Industrial and Systems Engineering Research Conference (ISERC), May 2013.

 Chimmalgi, Raj, & Knapp, G.M., "Detecting Controversial Topic Trends in Social Media", published abstract & presentation, 2013 Industrial and Systems Engineering Research Conference (ISERC), May 2013.

Weldu, Y., & Knapp, G.M., "BIM-Based Construction Schedule Generation and 4D Visualization", poster presentation, LSU Morehouse Scholars Conference, March 2013.

Javadpour, S.J., Halder, A., and Knapp, G.M., "Extracting Information from Business Documents Using Linguistic and Rule-Based System", Proceedings of the 2012 Industrial and Systems Engineering Research Conference (ISERC), May 2012.

Khazaeli, M., and Knapp, G.M., "Semantic Object Recognition for BIM Reverse Engineering", Proceedings of hte 2012 Industrial and Systems Engineering Research Conference (ISERC), May 2012.

Weldu, Y.W., and Knapp, G.M., "Automated Generation of 4D BIM Model through Spatial Reasoning", Proceedings of the Construction Research Conference (CRC) 2012, May 2012.

Calix, R.A., and Knapp, G.M., “Actor Level Emotion Magnitude Prediction in Text and Speech”, Multimedia Tools and Applications, http://dx.doi.org/10.1007/s11042-011-0909-8, 2011.

Calix, R.A., Khazaeli, M., Javadpour, S.J., and Knapp, G.M., “Dimensionality Reduction and Classification Analysis on the Audio Section of the SEMAINE Database”, First International Audio/Visual Emotion Challenge and Workshop (AVEC 2011), ACII 2011, Memphis, TN, October 9, 2011.

Calix, R.A., Javadpour, L.S., and Knapp, G.M., “Detection of Affective States from Text and Speech for Real-time Human Computer Interaction”, Human Factors, doi: 10.1177/0018720811425922, 2011. 

 Buddaraju, D., Harvey, C., and Knapp, G.M., “Performance of Control Room Operators in Alarm Management”, Proceedings of the 2011 Industrial Engineering Annual Research Conference (IERC), Reno, NV, May 21-25, 2011.

Calix, R. A., and Knapp, G. M.; “Affect Corpus 2.0: An Extension of a Corpus for Actor Level Emotion Magnitude Detection”, Proceedings of the 2nd ACM Multimedia Systems (MMSys) conference, Feb. 2011, San Jose, California, U.S.A. http://portal.acm.org/citation.cfm?id=1943570

Mallepudi, S. A., Calix, R. A., and Knapp, G. M.; "Material Classification and Automatic Content Enrichment of Images Using Supervised Learning and Knowledge Bases", Proceedings of the IS&T/SPIE International Conference on Multimedia Content Access: Algorithms and Systems V, January 2011, San Francisco, California. http://spie.org/x648.html?product_id=876583

Calix, R., Mallepudi, S., Knapp, G.M., and Nahmens, I.; "Factors that Influence Usage of Knowledge Management by Information Technology Professionals at Institutions of Higher Education", Journal of Management and Engineering Integration, Vol. 3, No. 1, Summer, 2010.

Calix, R.A., Mallepudi, S.A., Chen, B., and Knapp, G.M.; , "Emotion Recognition in Text for 3-D Facial Expression Rendering," IEEE Transactions on Multimedia, vol.12, no.6, pp.544-551, Oct. 2010. doi: 10.1109/TMM.2010.2052026.Link 

Faculty
Ph.D. Students
M.S. Students
Recent Grads