ISO/IEC JTC 1 SC 42 Artificial Intelligence - Working Group 4
Use Cases & Applications   04/25/2024

Editor's comments and enhancements are shown in green. [✓ Reviewed]

The quality of use case submissions will be evaluated for inclusion in the Working Group's Technical Report based on the application area, relevant AI technologies, credible reference sources (see References section), and the following characteristics:

  • [1] Data Focus & Learning: Use cases for AI system which utilizes Machine Learning, and those that use a fixed a priori knowledge base.
  • [2] Level of Autonomy: Use cases demonstrating several degrees (dependent, autonomous, human/critic in the loop, etc.) of AI system autonomy.
  • [3] Verifiability & Transparency: Use cases demonstrating several types and levels of verifiability and transparency, including approaches for explainable AI, accountability, etc.
  • [4] Impact: Use cases demonstrating the impact of AI systems to society, environment, etc.
  • [5] Architecture: Use cases demonstrating several architectural paradigms for AI systems (e.g., cloud, distributed AI, crowdsourcing, swarm intelligence, etc.)
  • [6] Functional aspects, trustworthiness, and societal concerns
  • [7] AI life cycle components include acquire/process/apply.
These characteristics are identified in red in the use case.

No. 43 ID: Use Case Name: Deep Learning Based User Intent Recognition
Application
Domain		Retail
Deployment
Model		On-premise systems
StatusIn operation
ScopeRecognizing users’ intent to solve their problems in e-commerce fields
Objective(s)To recognize and understand users’ intent by AI and deep learning technologies and apply such technologies to build chat bot systems to further reduce labor cost and to be applied in various fields.
Short
Description
(up to
150 words) 		Intelligent customer service chat bot is mainly used to categorize users’ questions, recognize users’ intents and answer users’ questions intelligently for different business jobs. Currently, this chat bot has been used to handle 90% of online customer service and has enabled JD.com to save over 100 million labor costs every year.
Complete Description JD.com has been committed to using technology to drive business growth and improve user experience in all customer service fields. Based on the improvement of customer consulting experience and the developing trend of artificial intelligence technology, as early as 2012, JD had decided to develop intelligent chat bots to fulfill the needs of continuous expansion of business, to save customer service costs and increase service capability. Intent recognition is a key and core technology to build such an intelligent customer service chat bot. By applying natural language processing technologies, deep learning technologies, traditional machine learning algorithms, intent recognition accuracy has reached to 95%. Based on accurate intents, and a series of solution finding algorithms, our chat bot can solve the user’s problems to a great extent and give the user a high quality consulting experience. Finally, in order to provide diversified and personalized customer services, we are continuously improving the accuracy of intent recognition, personalized solution generation, sentiment recognition, and image recognition. So far, intelligent customer service has revolutionized the traditional customer service consulting business.
Stakeholdersusers
Stakeholders'
Assets, Values
Systems'
Threats &
Vulnerabilities		high semantic ambiguity, Multiple language expressions in one sentence
Performance
Indicators (KPIs) 		Seq. No. Name Description Reference to mentioned
use case objectives
1 Accuracy The number of correctly recognized users’ intent over total number of users. Currently, accuracy reaches 95%. Improve accuracy of recognizing users’ intent
2 Resolution The number of answers solved over total number of questions asked Improve the resolution of questions from users
3 Satisfaction The number of users who are satisfied with customer service over total number of users Improve user experience
AI Features Task(s)Natural language processing
Method(s)Machine learning and deep learning
HardwareGPU and CPU
TopologyTensorFlow
Terms &
Concepts Used		Natural language processing, deep learning, CNN, HAN, logistic regression
Standardization
Opportunities
Requirements		Process Standardization will Improve Quality and Productivity
Challenges
& Issues		Current challenges of deep leaning and intent recognition:
  • high semantic ambiguity, similar sentences can deliver different meanings.
  • Unclear classification rules caused by complicated business logics
  • Hard to answer reasoning questions
Societal Concerns Description
  • Solve problems intelligently to increase efficiency
  • Free labors from repetitive work to save large amount of resources for the society
SDGs to
be achieved 		Decent work and economic growth
Data Characteristics
Description Question answering data from the JD.com online dialogue log
Source Customer's dialogue log at JD.com
Type Text
Volume (size) Millions
Velocity Real time
Variety various scenarios, various business, various categories of products
Variability
(rate of change) 		Non-linear
Quality good
Scenario Conditions
No. Scenario
Name 		Scenario
Description 		Triggering Event Pre-condition Post-Condition
1 Training Based on millions of labeled streaming data, train a model using diversified algorithms, such as a deep learning neural network or a traditional machine learning algorithm
2 Evaluation Evaluate the performance of the model on online dialogue data Each requirement must be satisfied or exceeded to reach the condition of 'success' (e.g. the accuracy should be more than 95%)
3 Execution Apply the trained model to predict user’s intent
4 Retraining Take a training sample from online dialogue to retrain the model and compare it with the old one by AB test The requirement is that the new model must be better than the old one
Trainng Scenario Name:  
Step No. Event Name of
Process/Activity 		Primary
Actor 		Description of
Process/Activity 		Requirement
1 Raw data stored in the database Data extraction Database engineer Extract related data from the database to generate the raw dataset
2 Completion of Step 1 Generating training samples Data labeling team Label the raw dataset of step one with 300 categories
3 Completion of Step 2 Pre-process AI engineer Segment the sentence into words and convert those words into vectors
Specification of training data After manual verifying, the accuracy of labelling should be more than 95%
Scenario Name Evaluation
Step No. Event Name of
Process/Activity 		Primary
Actor 		Description of
Process/Activity 		Requirement
1 Certain period of time has passed since the last training/retraining Data Extraction Database engineer Randomly take a sample from streaming data to form a test sample
2 Completion of Step 1 Prediction AI engineer Predict the test sample in step 1 by the trained model
3 Completion of Step 2 Evaluation Data labeling team Compare the result of predicted with the result of labeling
Input of Evaluation the result of labeling and the result of prediction
Output of Evaluation
Scenario Name Execution
Step No. Event Name of
Process/Activity 		Primary
Actor 		Description of
Process/Activity 		Requirement
1 Acquire the user’s query pre-process AI engineer pre-process The trained model has been in operation
2 Completion of Step 1 Text classification AI engineer Text classification
3 Completion of Step 2 Response AI trainer Response
Input of Execution
Output of Execution
Scenario Name Retraining
Step No. Event Name of
Process/Activity 		Primary
Actor 		Description of
Process/Activity 		Requirement
1 Certain period of time has passed since the last training/retraining Data extraction Database engineer
2 Completion of Step 1 Labeling the sample Data labeling team
3 Completion of Step 2 Model training AI engineer
4 Completion of Step 3 AB Test AI engineer The performance of the new model is better than results of the old one
5 Completion of Step 4 Online active of new model AI engineer
Specification of retraining data
References
No. Type Reference Status Impact of
use case 		Originator
Organization 		Link
1 Paper Convolutional Neural Networks for Sentence Classification New York University Link
2 Paper Hierarchical Attention Networks for Document Classification Carnegie Mellon University, Microsoft Research, Redmond Link
3 Paper LIBLINEAR: A library for large inear classification Journal of Machine Learning Research National Taiwan University Link

  • Peer-reviewed scientific/technical publications on AI applications (e.g. [1]).
  • Patent documents describing AI solutions (e.g. [2], [3]).
  • Technical reports or presentations by renowned AI experts (e.g. [4])
  • High quality company whitepapers and presentations
  • Publicly accessible sources with sufficient detail

    This list is not exhaustive. Other credible sources may be acceptable as well.

    Examples of credible sources:

    [1] B. Du Boulay. "Artificial Intelligence as an Effective Classroom Assistant". IEEE Intelligent Systems, V 31, p.76-81. 2016.

    [2] S. Hong. "Artificial intelligence audio apparatus and operation method thereof". N US 9,948,764, Available at: https://patents.google.com/patent/US20150120618A1/en. 2018.

    [3] M.R. Sumner, B.J. Newendorp and R.M. Orr. "Structured dictation using intelligent automated assistants". N US 9,865,280, 2018.

    [4] J. Hendler, S. Ellis, K. McGuire, N. Negedley, A. Weinstock, M. Klawonn and D. Burns. "WATSON@RPI, Technical Project Review".
    URL: https://www.slideshare.net/jahendler/watson-summer-review82013final. 2013