ISO/IEC JTC 1 SC 42 Artificial Intelligence - Working Group 4
Use Cases & Applications 03/29/2024
Editor's comments and enhancements are shown in green. [✓ Reviewed]
The quality of use case submissions will be evaluated for inclusion in the Working Group's Technical Report based on the application area, relevant AI technologies, credible reference sources (see References section), and the following characteristics:
[1] Data Focus & Learning: Use cases for AI system which utilizes Machine Learning, and those that use a fixed a priori knowledge base.
[2] Level of Autonomy: Use cases demonstrating several degrees (dependent, autonomous, human/critic in the loop, etc.) of AI system autonomy.
[3] Verifiability & Transparency: Use cases demonstrating several types and levels of verifiability and transparency, including approaches for explainable AI, accountability, etc.
[4] Impact: Use cases demonstrating the impact of AI systems to society, environment, etc.
[5] Architecture: Use cases demonstrating several architectural paradigms for AI systems (e.g., cloud, distributed AI, crowdsourcing, swarm intelligence, etc.)
[6] Functional aspects, trustworthiness, and societal concerns
[7] AI life cycle components include acquire/process/apply.
These characteristics are identified in red in the use case.
No.
13
ID:
Use Case Name:
AI solution to identify automatically false positives from a specific check for "untranslated target segments”"from an automated quality assurance tool
The scope of this use case is limited to automated linguistic quality assurance tools, but the outcome of this use case could be applicable to other areas, such as for example: Machine Translation, automated post-editing, Computer Aided Translation Analysis and pre-translation, etc. This use case will be relevant for contents across any domain.
To reduce the number of false positive issues for check for untranslated target segment for bilingual content with in-house automated quality assurance tool.
Short Description (up to 150 words)
In the future, we aim to build an AI solution that could automatically identify likely false positives issues from the results of the "check for untranslated target segments" following an approach where we could use machine learning based on already identified false positives by our users. The expected outcome would be to increase end user’s productivity when reviewing automated quality assurance findings and to change user behaviour to pay more attention to this type of issues by reducing the number of false positives in 80%. In addition, we would like to reduce the amount of time, we spent on a yearly basis on refining this check manually based on users' feedback.
Complete Description
Untranslated target segments contain characters, symbols, and words that remain the same in source and target language. These segments can contain, numbers, alphanumeric content, numbers, code, e-mail addresses, prices, proper nouns, etc. or any combination of those. On a yearly basis, this check produces over 1 Million potential issues across over 50 different languages. Refining this check manually based on annotated false positive data for each specific customer and product and for specific language pairs is very costly, and the coverage is never sufficient, as new content is constantly produced and there are always new opportunities for refining this check via code. In addition, because of the high proportion of false positives over (95.5%) our translators tend to ignore the output from this valuable check and in many cases, we suspect that valid relevant issues for situations when there are real forgotten translations are missed. There are typically three types of false positives for this type of check: 1) Language specific false positives, for example for situations where source and target segment need to be the same as the words from these segments are "cognates" with the same meaning. For example: Fig.1 2) Customer profile specific false positives, for example situations where certain segments are to be left untranslated based on specific guidelines from the customer, for example for segments that jut consist of Company names, Product Names or specific words and segments that have been determined as not to be translated by our customer: Fig.2 3) Segments that remain the same in source and target, because they act as special type of entities with some special meaning, for example: alphanumeric segments, for example part numbers, placeholders, code. Fig.3 The idea is to create an AI solution that can automatically identify results from the "check for untranslated target segment" that are likely to be a False Positive. With this solution, we expect to reduce the number of potential issues presented by this check to our end users in 80%. This way our end users can focus their efforts on those potential issues that are more likely to be valid corrections because there could have been a forgotten translation. In addition, we will be able to increase the productivity of our end users when reviewing automated quality assurance potential issues from their bilingual content evaluation, and we will be able to save costs internally as we won't have to manually implement code changes in this check based on manual analysis of our data based on user's annotation.
Bias from changes in requirements on the customer’s end or inappropriate training data.
Performance Indicators (KPIs)
Seq. No.
Name
Description
Reference to mentioned use case objectives
1
Coverage
Ratio of potential issues which are "of interest" for human evaluation. Ideal target is to reduce the current volume by 80%.
Improve accuracy
1
Precision
Correctly Predicted Anomalous scenarios/ Total Anomalous scenarios predicted
1
Features related to adulterants in radio spectrum
Intensities around NIR range
1
Average vehicle driving speed
Average vehicle driving speed on all the road sections in a given region
Improve the road utilization efficiency
1
Coverage
Ratio of EMR QC requirements done in the solution/all issued EMR QC requirements in China. Ideal target is 100%.
Improve accuracy
1
Accuracy
The number of correctly recognized users’ intent over total number of users. Currently, accuracy reaches 95%.
Improve accuracy of recognizing users’ intent
1
Classifier Accuracy
Without straightening and pre-processing, the average classification accuracy obtained was 68.5%. However, with preprocessing, the classification accuracy improved to 86.7%. These results are very likely to improve with more annotated training data for classification.
1
Closeness to Golden Batch
How close a process is to the best possible batch
Helps in isolation of bad batches from good batches by identifying combination of process variable trajectories that lead to good or bad batch operation.
1
Model Accuracy
Accuracy of the prediction model
The extent to which the setpoints have correctly predicted
1
Ratio of ML discovered failure rate to nominal failure rate
What combination of manufacturing processes/decisions leads to higher failure rates compared to nominal failure rate
Actionable intelligence to improve the manufacturing process of HV circuit breakers
1
Number of labors
reduced % of labors
improvement of productivity
1
Customer Satisfaction
The ratio of customer satisfaction when using this system for requests. The expectation is 100%
Increasing its ratio as high as possible
1
MIoU (Mean Intersection over Union)
The intersection of prediction area and actual area divided by the union of the predicted area and the actual area. Ideal target is 100%.
Improve accuracy
1
Classification Ratio
Real to Pseudo wrong classification
Establishes the quality of identification
1
Ease of use
Simplicity and efficiency during initial learning. Teaching process should be easy.
1
Zone of Influence/ Thermal Correlation Index
Extent of influence of ACUs on data center racks.
Helps in improved control.
1
Invisible Loss Time
Indicates the lost time of the asset in being idle or off or unplanned downtime
Asset Utilization Reports indicate the effectively utilized time there indicating the lost time and their causes
1
Prediction Accuracy
To what extent has the model been able to predict correctly
Provided ability as to % of times the quality complied
1
Algorithm accuracy
Output when compared to the human expert analysis of the same data
See Reference
1
Generation of Activities (land use information and time of travel)
Purpose of activities is assigned based on land use information and time of travel. Census data and national/ local travel surveys will provide validation for the process
Phrase 1
1
accuracy
The accuracy of infraction and incident detection from traffic pictures/videos
To increase the accuracy of traffic monitoring and inspection
2
Split
Proportion of the potential issues which are "more likely to be a valid issue" for our end users.
Average vehicle waiting time at all the intersections in a given region
Improve the road utilization efficiency
2
Resolution
The number of answers solved over total number of questions asked
Improve the resolution of questions from users
2
Annotation Completeness
35.9 chromosomes segmented out after crowd annotation, for 50 images having 46 chromosomes
2
% Reduction in Calibration Time
The amount of time saved from manually setting the calibration
2
Number of complaints
reduced % of labor's complaint
improvement of productivity
2
Accuracy
Among all the predicted customer sentiment classification, the ratio of accurate prediction, current value is 76.4%
Increasing to 90%
2
FAR (false acceptance rate)
Negative samples are identified as positive samples / Total number of negative samples.The low FAR, the more smartphone will get correct scenes and objects
Improve accuracy
2
Training efficiency
Amount of necessary data for training might lead to practical obstacles in application.
2
Overall drilling time
The time spent on one drilling job inclusive of the all downtimes
Real Time visibility into operations gives the operations early warnings to take actions immediately.
2
Generation of agents (travel times, speed on link)
Agents generated will build up in the network creating realistic conditionsw of congestion. Speed on links.
Phrase 2
2
split
Proportion of images requiring human inspection. The less the split, the higher the efficiency.
To minimize the human effort in inspection
3
Satisfaction
The number of users who are satisfied with customer service over total number of users
Improve user experience
3
Lead time
time from order to shipment
improvement of productivity
3
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 90%
Increasing to 90%
3
Initial success rate
After initial training, the success rate needs to be acceptable such that the system can be put in the production line.
3
Opeartion of service (number of users for the service)
Optimisation of route and operation time in the day. Validation provided using data collected by Mobility service operators during the operation of service
Phrase 3
3
resource utilization ratio
Achievable resource utilization ratio in the hardware infrastructure ( the higher the utilization ratio, the lower amount the required resource)
To reduce the infrastructure investment and overall solution cost
4
Accuracy
Among all the predicted customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
4
Speed of improvement
Higher convergence speed of the reinforcement algorithm is making the solution more attractive.
5
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
5
Operational efficiency
Cycle time is the primary measure in manufacturing industry.
6
Success rate
Very high success rate is required for the solution to be accepted.
Challenges: Try to achieve eventually 80% of the accuracy of linguists when identifying false positives for untranslated target segments, preventing as much as possible false negatives. Issues: segmentation of false positive data by Customer and Product profile could be challenging.
Editor's comments and enhancements are shown in green. [✓ Reviewed]
The quality of use case submissions will be evaluated for inclusion in the Working Group's Technical Report based on the application area, relevant AI technologies, credible reference sources (see References section), and the following characteristics:
[1] Data Focus & Learning: Use cases for AI system which utilizes Machine Learning, and those that use a fixed a priori knowledge base.
[2] Level of Autonomy: Use cases demonstrating several degrees (dependent, autonomous, human/critic in the loop, etc.) of AI system autonomy.
[3] Verifiability & Transparency: Use cases demonstrating several types and levels of verifiability and transparency, including approaches for explainable AI, accountability, etc.
[4] Impact: Use cases demonstrating the impact of AI systems to society, environment, etc.
[5] Architecture: Use cases demonstrating several architectural paradigms for AI systems (e.g., cloud, distributed AI, crowdsourcing, swarm intelligence, etc.)
[6] Functional aspects, trustworthiness, and societal concerns
[7] AI life cycle components include acquire/process/apply.
These characteristics are identified in red in the use case.
No.
45
ID:
Use Case Name:
Anomaly Detection in Sensor Data Using Deep Learning techniques
Identify Anomalies and Events by learning the temporal patterns of sensor data, based on Deep Learning techniques
Short Description (up to 150 words)
Mechanical devices such as engines, vehicles, aircrafts, etc., are typically instrumented with numerous sensors to capture the behaviour and health of the machine. The sensors temporal data has several complex patterns that are very hard to identify with traditional methods. We have proposed the use of Deep Learning algorithms for analysing such temporal patterns for anomaly/event detection, diagnosis, root cause analysis. Algorithms proposed so far are LSTM-AD, EncDec-AD, online RNN-AD. We used industrial datasets wherever possible and publically available datasets in other scenarios. In most of the cases, our algorithms were significantly better than other methods.
Complete Description
Mechanical devices such as engines, vehicles, aircrafts, etc., are typically instrumented with numerous sensors to capture the behaviour and health of the machine. However, there are often external factors or variables which are not captured by sensors leading to time-series which are inherently unpredictable. For instance, manual controls and/or unmonitored environmental conditions or load may lead to inherently unpredictable time-series. Detecting anomalies/events in such scenarios becomes challenging using standard approaches based on mathematical models that rely on stationarity, or prediction models that utilize prediction errors to detect anomalies. LSTM-AD Our Work started with Stacked LSTM network which is trained on non-anomalous data and used as a predictor over a number of time steps. The resulting prediction errors are modeled as a multivariate Gaussian distribution, which is used to assess the likelihood of anomalous behavior. The efficacy of this approach was demonstrated on four datasets: ECG, space shuttle, power demand, and multi-sensor engine dataset. EncDec-AD As an extension to the prior work we proposed a Long Short Term Memory Networks based Encoder-Decoder scheme for Anomaly Detection (EncDec-AD) that learns to reconstruct normal time-series behavior, and thereafter uses reconstruction error to detect anomalies. We experimented with three publicly available quasi predictable time-series datasets: power demand, space shuttle, and ECG, and two real-world engine datasets with both predictive and unpredictable behavior. We had shown that EncDec-AD is robust and can detect anomalies from predictable, unpredictable, periodic, aperiodic, and quasi-periodic time-series. Further, we showed that EncDec-AD is able to detect anomalies from short time-series (length as small as 30) as well as long time-series (length as large as 500). Online-AD The common approach of training one model in an offline manner using historical data is likely to fail under dynamically changing and non-stationary environments where the definition of normal behavior changes over time making the model irrelevant and ineffective. We described a temporal model based on Recurrent Neural Networks (RNNs) for time series anomaly detection to address challenges posed by sudden or regular changes in normal behaviour. The model is trained incrementally as new data becomes available, and is capable of adapting to the changes in the data distribution. RNN is used to make multi-step predictions of the time series, and the prediction errors are used to update the RNN model as well as detect anomalies and change points. Large prediction error is used to indicate anomalous behaviour or a change (drift) in normal behaviour. Further, the prediction errors are also used to update the RNN model in such a way that short term anomalies or outliers do not lead to a drastic change in the model parameters whereas high prediction errors over a period of time lead to significant updates in the model parameters such that the model rapidly adapts to the new norm. We demonstrate the efficacy of the proposed approach on a diverse set of synthetic, publicly available and proprietary real-world datasets.
Ratio of potential issues which are "of interest" for human evaluation. Ideal target is to reduce the current volume by 80%.
Improve accuracy
1
Precision
Correctly Predicted Anomalous scenarios/ Total Anomalous scenarios predicted
1
Features related to adulterants in radio spectrum
Intensities around NIR range
1
Average vehicle driving speed
Average vehicle driving speed on all the road sections in a given region
Improve the road utilization efficiency
1
Coverage
Ratio of EMR QC requirements done in the solution/all issued EMR QC requirements in China. Ideal target is 100%.
Improve accuracy
1
Accuracy
The number of correctly recognized users’ intent over total number of users. Currently, accuracy reaches 95%.
Improve accuracy of recognizing users’ intent
1
Classifier Accuracy
Without straightening and pre-processing, the average classification accuracy obtained was 68.5%. However, with preprocessing, the classification accuracy improved to 86.7%. These results are very likely to improve with more annotated training data for classification.
1
Closeness to Golden Batch
How close a process is to the best possible batch
Helps in isolation of bad batches from good batches by identifying combination of process variable trajectories that lead to good or bad batch operation.
1
Model Accuracy
Accuracy of the prediction model
The extent to which the setpoints have correctly predicted
1
Ratio of ML discovered failure rate to nominal failure rate
What combination of manufacturing processes/decisions leads to higher failure rates compared to nominal failure rate
Actionable intelligence to improve the manufacturing process of HV circuit breakers
1
Number of labors
reduced % of labors
improvement of productivity
1
Customer Satisfaction
The ratio of customer satisfaction when using this system for requests. The expectation is 100%
Increasing its ratio as high as possible
1
MIoU (Mean Intersection over Union)
The intersection of prediction area and actual area divided by the union of the predicted area and the actual area. Ideal target is 100%.
Improve accuracy
1
Classification Ratio
Real to Pseudo wrong classification
Establishes the quality of identification
1
Ease of use
Simplicity and efficiency during initial learning. Teaching process should be easy.
1
Zone of Influence/ Thermal Correlation Index
Extent of influence of ACUs on data center racks.
Helps in improved control.
1
Invisible Loss Time
Indicates the lost time of the asset in being idle or off or unplanned downtime
Asset Utilization Reports indicate the effectively utilized time there indicating the lost time and their causes
1
Prediction Accuracy
To what extent has the model been able to predict correctly
Provided ability as to % of times the quality complied
1
Algorithm accuracy
Output when compared to the human expert analysis of the same data
See Reference
1
Generation of Activities (land use information and time of travel)
Purpose of activities is assigned based on land use information and time of travel. Census data and national/ local travel surveys will provide validation for the process
Phrase 1
1
accuracy
The accuracy of infraction and incident detection from traffic pictures/videos
To increase the accuracy of traffic monitoring and inspection
2
Split
Proportion of the potential issues which are "more likely to be a valid issue" for our end users.
Average vehicle waiting time at all the intersections in a given region
Improve the road utilization efficiency
2
Resolution
The number of answers solved over total number of questions asked
Improve the resolution of questions from users
2
Annotation Completeness
35.9 chromosomes segmented out after crowd annotation, for 50 images having 46 chromosomes
2
% Reduction in Calibration Time
The amount of time saved from manually setting the calibration
2
Number of complaints
reduced % of labor's complaint
improvement of productivity
2
Accuracy
Among all the predicted customer sentiment classification, the ratio of accurate prediction, current value is 76.4%
Increasing to 90%
2
FAR (false acceptance rate)
Negative samples are identified as positive samples / Total number of negative samples.The low FAR, the more smartphone will get correct scenes and objects
Improve accuracy
2
Training efficiency
Amount of necessary data for training might lead to practical obstacles in application.
2
Overall drilling time
The time spent on one drilling job inclusive of the all downtimes
Real Time visibility into operations gives the operations early warnings to take actions immediately.
2
Generation of agents (travel times, speed on link)
Agents generated will build up in the network creating realistic conditionsw of congestion. Speed on links.
Phrase 2
2
split
Proportion of images requiring human inspection. The less the split, the higher the efficiency.
To minimize the human effort in inspection
3
Satisfaction
The number of users who are satisfied with customer service over total number of users
Improve user experience
3
Lead time
time from order to shipment
improvement of productivity
3
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 90%
Increasing to 90%
3
Initial success rate
After initial training, the success rate needs to be acceptable such that the system can be put in the production line.
3
Opeartion of service (number of users for the service)
Optimisation of route and operation time in the day. Validation provided using data collected by Mobility service operators during the operation of service
Phrase 3
3
resource utilization ratio
Achievable resource utilization ratio in the hardware infrastructure ( the higher the utilization ratio, the lower amount the required resource)
To reduce the infrastructure investment and overall solution cost
4
Accuracy
Among all the predicted customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
4
Speed of improvement
Higher convergence speed of the reinforcement algorithm is making the solution more attractive.
5
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
5
Operational efficiency
Cycle time is the primary measure in manufacturing industry.
6
Success rate
Very high success rate is required for the solution to be accepted.
Editor's comments and enhancements are shown in green. [✓ Reviewed]
The quality of use case submissions will be evaluated for inclusion in the Working Group's Technical Report based on the application area, relevant AI technologies, credible reference sources (see References section), and the following characteristics:
[1] Data Focus & Learning: Use cases for AI system which utilizes Machine Learning, and those that use a fixed a priori knowledge base.
[2] Level of Autonomy: Use cases demonstrating several degrees (dependent, autonomous, human/critic in the loop, etc.) of AI system autonomy.
[3] Verifiability & Transparency: Use cases demonstrating several types and levels of verifiability and transparency, including approaches for explainable AI, accountability, etc.
[4] Impact: Use cases demonstrating the impact of AI systems to society, environment, etc.
[5] Architecture: Use cases demonstrating several architectural paradigms for AI systems (e.g., cloud, distributed AI, crowdsourcing, swarm intelligence, etc.)
[6] Functional aspects, trustworthiness, and societal concerns
[7] AI life cycle components include acquire/process/apply.
These characteristics are identified in red in the use case.
No.
19
ID:
Use Case Name:
AI to understand adulteration in commonly used food items
To device a simple , cost effective tool to identify the adulteration in food items at point of purchase
Short Description (up to 150 words)
Food adulteration is one of the big evil of modern society. Hyperspectral technology was evaluated to find out adulteration in food items
Complete Description
Food adulteration is becoming menace specially with adulterants that are either carcinogenic or harmful to body parts like kidney. To give few examples, Milk is adulterated with Soda, Urea and detergents. Whereas mangoes and bananas are quickly ripened by calcium carbide and so on. Common man cannot live without these items. There is no frugal way to identify these type of adulterations. Experiment of controlled adulteration was done and hyperspectral reflectance reading were taken. AI helped to find the patterns in hyperspectral signature and was able to reliably classify ( 90% ++) samples that were unadulterated and adulterated.
Ratio of potential issues which are "of interest" for human evaluation. Ideal target is to reduce the current volume by 80%.
Improve accuracy
1
Precision
Correctly Predicted Anomalous scenarios/ Total Anomalous scenarios predicted
1
Features related to adulterants in radio spectrum
Intensities around NIR range
1
Average vehicle driving speed
Average vehicle driving speed on all the road sections in a given region
Improve the road utilization efficiency
1
Coverage
Ratio of EMR QC requirements done in the solution/all issued EMR QC requirements in China. Ideal target is 100%.
Improve accuracy
1
Accuracy
The number of correctly recognized users’ intent over total number of users. Currently, accuracy reaches 95%.
Improve accuracy of recognizing users’ intent
1
Classifier Accuracy
Without straightening and pre-processing, the average classification accuracy obtained was 68.5%. However, with preprocessing, the classification accuracy improved to 86.7%. These results are very likely to improve with more annotated training data for classification.
1
Closeness to Golden Batch
How close a process is to the best possible batch
Helps in isolation of bad batches from good batches by identifying combination of process variable trajectories that lead to good or bad batch operation.
1
Model Accuracy
Accuracy of the prediction model
The extent to which the setpoints have correctly predicted
1
Ratio of ML discovered failure rate to nominal failure rate
What combination of manufacturing processes/decisions leads to higher failure rates compared to nominal failure rate
Actionable intelligence to improve the manufacturing process of HV circuit breakers
1
Number of labors
reduced % of labors
improvement of productivity
1
Customer Satisfaction
The ratio of customer satisfaction when using this system for requests. The expectation is 100%
Increasing its ratio as high as possible
1
MIoU (Mean Intersection over Union)
The intersection of prediction area and actual area divided by the union of the predicted area and the actual area. Ideal target is 100%.
Improve accuracy
1
Classification Ratio
Real to Pseudo wrong classification
Establishes the quality of identification
1
Ease of use
Simplicity and efficiency during initial learning. Teaching process should be easy.
1
Zone of Influence/ Thermal Correlation Index
Extent of influence of ACUs on data center racks.
Helps in improved control.
1
Invisible Loss Time
Indicates the lost time of the asset in being idle or off or unplanned downtime
Asset Utilization Reports indicate the effectively utilized time there indicating the lost time and their causes
1
Prediction Accuracy
To what extent has the model been able to predict correctly
Provided ability as to % of times the quality complied
1
Algorithm accuracy
Output when compared to the human expert analysis of the same data
See Reference
1
Generation of Activities (land use information and time of travel)
Purpose of activities is assigned based on land use information and time of travel. Census data and national/ local travel surveys will provide validation for the process
Phrase 1
1
accuracy
The accuracy of infraction and incident detection from traffic pictures/videos
To increase the accuracy of traffic monitoring and inspection
2
Split
Proportion of the potential issues which are "more likely to be a valid issue" for our end users.
Average vehicle waiting time at all the intersections in a given region
Improve the road utilization efficiency
2
Resolution
The number of answers solved over total number of questions asked
Improve the resolution of questions from users
2
Annotation Completeness
35.9 chromosomes segmented out after crowd annotation, for 50 images having 46 chromosomes
2
% Reduction in Calibration Time
The amount of time saved from manually setting the calibration
2
Number of complaints
reduced % of labor's complaint
improvement of productivity
2
Accuracy
Among all the predicted customer sentiment classification, the ratio of accurate prediction, current value is 76.4%
Increasing to 90%
2
FAR (false acceptance rate)
Negative samples are identified as positive samples / Total number of negative samples.The low FAR, the more smartphone will get correct scenes and objects
Improve accuracy
2
Training efficiency
Amount of necessary data for training might lead to practical obstacles in application.
2
Overall drilling time
The time spent on one drilling job inclusive of the all downtimes
Real Time visibility into operations gives the operations early warnings to take actions immediately.
2
Generation of agents (travel times, speed on link)
Agents generated will build up in the network creating realistic conditionsw of congestion. Speed on links.
Phrase 2
2
split
Proportion of images requiring human inspection. The less the split, the higher the efficiency.
To minimize the human effort in inspection
3
Satisfaction
The number of users who are satisfied with customer service over total number of users
Improve user experience
3
Lead time
time from order to shipment
improvement of productivity
3
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 90%
Increasing to 90%
3
Initial success rate
After initial training, the success rate needs to be acceptable such that the system can be put in the production line.
3
Opeartion of service (number of users for the service)
Optimisation of route and operation time in the day. Validation provided using data collected by Mobility service operators during the operation of service
Phrase 3
3
resource utilization ratio
Achievable resource utilization ratio in the hardware infrastructure ( the higher the utilization ratio, the lower amount the required resource)
To reduce the infrastructure investment and overall solution cost
4
Accuracy
Among all the predicted customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
4
Speed of improvement
Higher convergence speed of the reinforcement algorithm is making the solution more attractive.
5
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
5
Operational efficiency
Cycle time is the primary measure in manufacturing industry.
6
Success rate
Very high success rate is required for the solution to be accepted.
Adulterated milk is hazard for children, many aliments including cancer / kidney failures due to consumption of adulterated food. If the AI system is rolled out and taken as reliable then it should be able to perform in all cases and scenarios.
Editor's comments and enhancements are shown in green. [✓ Reviewed]
The quality of use case submissions will be evaluated for inclusion in the Working Group's Technical Report based on the application area, relevant AI technologies, credible reference sources (see References section), and the following characteristics:
[1] Data Focus & Learning: Use cases for AI system which utilizes Machine Learning, and those that use a fixed a priori knowledge base.
[2] Level of Autonomy: Use cases demonstrating several degrees (dependent, autonomous, human/critic in the loop, etc.) of AI system autonomy.
[3] Verifiability & Transparency: Use cases demonstrating several types and levels of verifiability and transparency, including approaches for explainable AI, accountability, etc.
[4] Impact: Use cases demonstrating the impact of AI systems to society, environment, etc.
[5] Architecture: Use cases demonstrating several architectural paradigms for AI systems (e.g., cloud, distributed AI, crowdsourcing, swarm intelligence, etc.)
[6] Functional aspects, trustworthiness, and societal concerns
[7] AI life cycle components include acquire/process/apply.
These characteristics are identified in red in the use case.
No.
48
ID:
Use Case Name:
Value-based Service
Application Domain
Manufacturing
Deployment Model
Hybrid deployment: Cloud and on-premise deployment in the production field
The objective of this use case is the provision of remote services for product and production based on (generic) service platforms. This use case can be seen as a fundament for the deployment of arbitrary AI remote services.
Short Description (up to 150 words)
Service platforms collects data from product use – for example machines or plants – and analyses and processes this data to provide tailor-made individualized services, e.g. optimized maintenance at the proper time, or the timely provision of the correct process parameters for a production task currently being requested. Companies offering these services (service providers) occupy the interface between the product provider and the user.
Complete Description
Use Case description taken from [1,2,3]. In the consumer area, the increased interconnectivity of users which has made it possible to collect user data has made a whole new range of services possible. For example, navigation systems in our cars not only determine the shortest route, but also the quickest, as the traffic situation is assessed in real time based on movement data from other users. Entertainment media is no longer purchased rather made available as needed using streaming services. The services offered extend beyond simply making the products available. The individual customer receives optimized offers, based on user data: the quickest route during rush hour, or music tailored to that customer’s taste. Similar developments are occurring in an increasingly interconnected industrial environment. Services that go significantly beyond simply providing a production unit – a contemporary example is leasing – are gaining in importance and are changing the classic value-added processes and business models.
Key aspects At the heart of this application scenario are IT platforms that collect data from product use – for example machines or plants for production purposes – and analyze and process this data to provide tailor-made individualized services. This could include for example optimized maintenance at the proper time, or the timely provision of the correct process parameters for a production task currently being requested. The collected data could be product parameters, for example the machines and plants required for manufacture, the product status information, or data from the production process or the upstream supply process. Even the characteristics of the processed raw materials or the parts of the product could be included. The goal is to use this data as a raw material for optimizing products and production processes and for new services. This can help to not only improve existing value chains but also perhaps create new value-added elements.
Effect on value chains The industrial environment today is influenced in principle by two actors – the product provider (i.e. manufacturers of production facilities and service providers) and the customer (product users, i.e. production facility operators), who work together with varying degrees of intensity.
With the introduction of Value-Based Services an additional actor enters the scene, operating IT platforms that it uses to provide new services to both classic partners. This platform operator could be a new element of the value chain, that is, an autonomous company. However, this role could be taken on by product providers by increasing their value added compared with the current situation.
Product providers make their product data and parameters available. On the basis of all of this user data, new services can now be developed, such as individual optimized maintenance or specific operating and process parameters that optimize or even expand production capabilities of the existing infrastructure. The companies offering these services (service providers) occupy the interface between the product provider and the user. The result is that the share in the value chain spanning from the product provider to the user can be shifted significantly, compared with the situation today. The user can then distinguish between the products by considering the accompanying services or the possibility of expanding those services even after purchasing the product, and no longer primarily by the (physical) specifications mandated by the product provider. This makes it very attractive for the product provider to use such platforms and to offer new services on them.
Value added for participants In this application scenario the value added for the product provider stems from the availability of a multitude of process data from various application scenarios, which the user can apply to further development of its product port-folio. As an operator of related IT platforms, the product provider can offer new services. In this way, it strengthens customer loyalty and increases its portion of value added.
Ratio of potential issues which are "of interest" for human evaluation. Ideal target is to reduce the current volume by 80%.
Improve accuracy
1
Precision
Correctly Predicted Anomalous scenarios/ Total Anomalous scenarios predicted
1
Features related to adulterants in radio spectrum
Intensities around NIR range
1
Average vehicle driving speed
Average vehicle driving speed on all the road sections in a given region
Improve the road utilization efficiency
1
Coverage
Ratio of EMR QC requirements done in the solution/all issued EMR QC requirements in China. Ideal target is 100%.
Improve accuracy
1
Accuracy
The number of correctly recognized users’ intent over total number of users. Currently, accuracy reaches 95%.
Improve accuracy of recognizing users’ intent
1
Classifier Accuracy
Without straightening and pre-processing, the average classification accuracy obtained was 68.5%. However, with preprocessing, the classification accuracy improved to 86.7%. These results are very likely to improve with more annotated training data for classification.
1
Closeness to Golden Batch
How close a process is to the best possible batch
Helps in isolation of bad batches from good batches by identifying combination of process variable trajectories that lead to good or bad batch operation.
1
Model Accuracy
Accuracy of the prediction model
The extent to which the setpoints have correctly predicted
1
Ratio of ML discovered failure rate to nominal failure rate
What combination of manufacturing processes/decisions leads to higher failure rates compared to nominal failure rate
Actionable intelligence to improve the manufacturing process of HV circuit breakers
1
Number of labors
reduced % of labors
improvement of productivity
1
Customer Satisfaction
The ratio of customer satisfaction when using this system for requests. The expectation is 100%
Increasing its ratio as high as possible
1
MIoU (Mean Intersection over Union)
The intersection of prediction area and actual area divided by the union of the predicted area and the actual area. Ideal target is 100%.
Improve accuracy
1
Classification Ratio
Real to Pseudo wrong classification
Establishes the quality of identification
1
Ease of use
Simplicity and efficiency during initial learning. Teaching process should be easy.
1
Zone of Influence/ Thermal Correlation Index
Extent of influence of ACUs on data center racks.
Helps in improved control.
1
Invisible Loss Time
Indicates the lost time of the asset in being idle or off or unplanned downtime
Asset Utilization Reports indicate the effectively utilized time there indicating the lost time and their causes
1
Prediction Accuracy
To what extent has the model been able to predict correctly
Provided ability as to % of times the quality complied
1
Algorithm accuracy
Output when compared to the human expert analysis of the same data
See Reference
1
Generation of Activities (land use information and time of travel)
Purpose of activities is assigned based on land use information and time of travel. Census data and national/ local travel surveys will provide validation for the process
Phrase 1
1
accuracy
The accuracy of infraction and incident detection from traffic pictures/videos
To increase the accuracy of traffic monitoring and inspection
2
Split
Proportion of the potential issues which are "more likely to be a valid issue" for our end users.
Average vehicle waiting time at all the intersections in a given region
Improve the road utilization efficiency
2
Resolution
The number of answers solved over total number of questions asked
Improve the resolution of questions from users
2
Annotation Completeness
35.9 chromosomes segmented out after crowd annotation, for 50 images having 46 chromosomes
2
% Reduction in Calibration Time
The amount of time saved from manually setting the calibration
2
Number of complaints
reduced % of labor's complaint
improvement of productivity
2
Accuracy
Among all the predicted customer sentiment classification, the ratio of accurate prediction, current value is 76.4%
Increasing to 90%
2
FAR (false acceptance rate)
Negative samples are identified as positive samples / Total number of negative samples.The low FAR, the more smartphone will get correct scenes and objects
Improve accuracy
2
Training efficiency
Amount of necessary data for training might lead to practical obstacles in application.
2
Overall drilling time
The time spent on one drilling job inclusive of the all downtimes
Real Time visibility into operations gives the operations early warnings to take actions immediately.
2
Generation of agents (travel times, speed on link)
Agents generated will build up in the network creating realistic conditionsw of congestion. Speed on links.
Phrase 2
2
split
Proportion of images requiring human inspection. The less the split, the higher the efficiency.
To minimize the human effort in inspection
3
Satisfaction
The number of users who are satisfied with customer service over total number of users
Improve user experience
3
Lead time
time from order to shipment
improvement of productivity
3
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 90%
Increasing to 90%
3
Initial success rate
After initial training, the success rate needs to be acceptable such that the system can be put in the production line.
3
Opeartion of service (number of users for the service)
Optimisation of route and operation time in the day. Validation provided using data collected by Mobility service operators during the operation of service
Phrase 3
3
resource utilization ratio
Achievable resource utilization ratio in the hardware infrastructure ( the higher the utilization ratio, the lower amount the required resource)
To reduce the infrastructure investment and overall solution cost
4
Accuracy
Among all the predicted customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
4
Speed of improvement
Higher convergence speed of the reinforcement algorithm is making the solution more attractive.
5
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
5
Operational efficiency
Cycle time is the primary measure in manufacturing industry.
6
Success rate
Very high success rate is required for the solution to be accepted.
AI Features
Task(s)
Reasoning and autonomous problem solving in the platform, services based on the platform use AI features, e.g. for predictive maintenance, data semantics (cf. [5,6] for an overview)
Standardization needs for setting up this use case is currently under further investigation. Some initial intentions on standardization needs are the following: For this use case, standardization can be seen as enabler because an agreement on a (small set of) communication protocols would facilitate to connect to the platform and use this protocol also for device2device communication. Since services running on a platform are not aware of an implicit sematic of data sources (machines, sensors, actuators, …), an explicit semantic or a common vocabulary is need describing data and enable reasoning about machine states on premise (on the machine/edge) as well as on the cloud. For cloud2cloud communication and cloud federation, further interoperability standards are required on communication level as well as on data semantics level.
Editor's comments and enhancements are shown in green. [✓ Reviewed]
The quality of use case submissions will be evaluated for inclusion in the Working Group's Technical Report based on the application area, relevant AI technologies, credible reference sources (see References section), and the following characteristics:
[1] Data Focus & Learning: Use cases for AI system which utilizes Machine Learning, and those that use a fixed a priori knowledge base.
[2] Level of Autonomy: Use cases demonstrating several degrees (dependent, autonomous, human/critic in the loop, etc.) of AI system autonomy.
[3] Verifiability & Transparency: Use cases demonstrating several types and levels of verifiability and transparency, including approaches for explainable AI, accountability, etc.
[4] Impact: Use cases demonstrating the impact of AI systems to society, environment, etc.
[5] Architecture: Use cases demonstrating several architectural paradigms for AI systems (e.g., cloud, distributed AI, crowdsourcing, swarm intelligence, etc.)
[6] Functional aspects, trustworthiness, and societal concerns
[7] AI life cycle components include acquire/process/apply.
These characteristics are identified in red in the use case.
No.
49
ID:
Use Case Name:
AI solution for traffic signal Optimization based on multi-source data fusion
Generate traffic signal timing plans by analyzing traffic flow status and patterns based on fusing internet data, induction coils data and video data, and control the traffic signal with the generated timing plans in a real-time, self-adaptive and cooperative way
To find an effective and efficient solution to improve the road utilization efficiency by increasing traffic flow speed and reducing traffic flow waiting time.
Short Description (up to 150 words)
An AI solution was developed that could recognize real-time traffic flow status and abstract traffic flow patterns by fusing internet data, induction coils data and video data, and could generate optimized traffic signal timing plan by self-adaptively responding to real-time traffic flow fluctuation and with regards to traffic flow coordination among multiple intersections within a given region.
Complete Description
By far, traffic administrator produces traffic signal timing plans by observing traffic flow situation on-site at intersections or through videos, and relies on her/his personal experience. Then, the timing plans are input into and executed by the traffic signal control system. The disadvantages of this manual traffic signal timing plan generation approach are as follows: 1. Low computing efficiency, it consumes very long time for traffic administrator to observe and analyze traffic patterns. 2. Low computing precision, traffic administrator only cares about the macro traffic flow tendency at intersections without computing detailed traffic parameters such as speed, queue length in each lane, etc. 3. Slow response to traffic flow fluctuation, it is hard for traffic administrator to produce adaptive timing plan in time with respect to real-time traffic flow fluctuation, due to her/his limited computing ability, not mention to coordinate traffic flows among multiple intersections by controlling the traffic signal in real-time. 4. Experienced traffic administrators are severely in short for cities with the scale of thousands intersections.
For solving the above problems, the AI provider applies a multi-source data fusion approach to recognize the traffic flow status and generalize the traffic flow pattern by analyzing the internet data (i.e., vehicle driving trajectory data provided by internet service supplier), detector data collected by induction coils, and structured data recognized from videos. Furthermore, the AI provider develops an optimization method to figure out optimized traffic signal timing plan by self-adaptively responding to real-time traffic flow fluctuation and with regards to traffic flow coordination among multiple intersections.
The developed methods have been applied in practice within a given region from a large city. It generates traffic signal timing plans for all the intersections in the region according to their real-time traffic flow fluctuation with an updating frequency of 5 minutes per time. Compared with the manual traffic signal timing plans form the traffic administrators, the plans generated by the new method have increased the average vehicle driving speed by 9%, and reduced the average vehicle waiting time by 15%.
Ratio of potential issues which are "of interest" for human evaluation. Ideal target is to reduce the current volume by 80%.
Improve accuracy
1
Precision
Correctly Predicted Anomalous scenarios/ Total Anomalous scenarios predicted
1
Features related to adulterants in radio spectrum
Intensities around NIR range
1
Average vehicle driving speed
Average vehicle driving speed on all the road sections in a given region
Improve the road utilization efficiency
1
Coverage
Ratio of EMR QC requirements done in the solution/all issued EMR QC requirements in China. Ideal target is 100%.
Improve accuracy
1
Accuracy
The number of correctly recognized users’ intent over total number of users. Currently, accuracy reaches 95%.
Improve accuracy of recognizing users’ intent
1
Classifier Accuracy
Without straightening and pre-processing, the average classification accuracy obtained was 68.5%. However, with preprocessing, the classification accuracy improved to 86.7%. These results are very likely to improve with more annotated training data for classification.
1
Closeness to Golden Batch
How close a process is to the best possible batch
Helps in isolation of bad batches from good batches by identifying combination of process variable trajectories that lead to good or bad batch operation.
1
Model Accuracy
Accuracy of the prediction model
The extent to which the setpoints have correctly predicted
1
Ratio of ML discovered failure rate to nominal failure rate
What combination of manufacturing processes/decisions leads to higher failure rates compared to nominal failure rate
Actionable intelligence to improve the manufacturing process of HV circuit breakers
1
Number of labors
reduced % of labors
improvement of productivity
1
Customer Satisfaction
The ratio of customer satisfaction when using this system for requests. The expectation is 100%
Increasing its ratio as high as possible
1
MIoU (Mean Intersection over Union)
The intersection of prediction area and actual area divided by the union of the predicted area and the actual area. Ideal target is 100%.
Improve accuracy
1
Classification Ratio
Real to Pseudo wrong classification
Establishes the quality of identification
1
Ease of use
Simplicity and efficiency during initial learning. Teaching process should be easy.
1
Zone of Influence/ Thermal Correlation Index
Extent of influence of ACUs on data center racks.
Helps in improved control.
1
Invisible Loss Time
Indicates the lost time of the asset in being idle or off or unplanned downtime
Asset Utilization Reports indicate the effectively utilized time there indicating the lost time and their causes
1
Prediction Accuracy
To what extent has the model been able to predict correctly
Provided ability as to % of times the quality complied
1
Algorithm accuracy
Output when compared to the human expert analysis of the same data
See Reference
1
Generation of Activities (land use information and time of travel)
Purpose of activities is assigned based on land use information and time of travel. Census data and national/ local travel surveys will provide validation for the process
Phrase 1
1
accuracy
The accuracy of infraction and incident detection from traffic pictures/videos
To increase the accuracy of traffic monitoring and inspection
2
Split
Proportion of the potential issues which are "more likely to be a valid issue" for our end users.
Average vehicle waiting time at all the intersections in a given region
Improve the road utilization efficiency
2
Resolution
The number of answers solved over total number of questions asked
Improve the resolution of questions from users
2
Annotation Completeness
35.9 chromosomes segmented out after crowd annotation, for 50 images having 46 chromosomes
2
% Reduction in Calibration Time
The amount of time saved from manually setting the calibration
2
Number of complaints
reduced % of labor's complaint
improvement of productivity
2
Accuracy
Among all the predicted customer sentiment classification, the ratio of accurate prediction, current value is 76.4%
Increasing to 90%
2
FAR (false acceptance rate)
Negative samples are identified as positive samples / Total number of negative samples.The low FAR, the more smartphone will get correct scenes and objects
Improve accuracy
2
Training efficiency
Amount of necessary data for training might lead to practical obstacles in application.
2
Overall drilling time
The time spent on one drilling job inclusive of the all downtimes
Real Time visibility into operations gives the operations early warnings to take actions immediately.
2
Generation of agents (travel times, speed on link)
Agents generated will build up in the network creating realistic conditionsw of congestion. Speed on links.
Phrase 2
2
split
Proportion of images requiring human inspection. The less the split, the higher the efficiency.
To minimize the human effort in inspection
3
Satisfaction
The number of users who are satisfied with customer service over total number of users
Improve user experience
3
Lead time
time from order to shipment
improvement of productivity
3
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 90%
Increasing to 90%
3
Initial success rate
After initial training, the success rate needs to be acceptable such that the system can be put in the production line.
3
Opeartion of service (number of users for the service)
Optimisation of route and operation time in the day. Validation provided using data collected by Mobility service operators during the operation of service
Phrase 3
3
resource utilization ratio
Achievable resource utilization ratio in the hardware infrastructure ( the higher the utilization ratio, the lower amount the required resource)
To reduce the infrastructure investment and overall solution cost
4
Accuracy
Among all the predicted customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
4
Speed of improvement
Higher convergence speed of the reinforcement algorithm is making the solution more attractive.
5
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
5
Operational efficiency
Cycle time is the primary measure in manufacturing industry.
6
Success rate
Very high success rate is required for the solution to be accepted.
Traffic signal self-adaptive and coordinative control for a large number of intersections. Issues: 1. Not all intersections are equipped with detectors such as induction coil or video. 2. The detectors may output abnormal values which need data clean processings
Standardization Opportunities Requirements
Challenges & Issues
Challenges:Traffic signal self-adaptive and coordinated control for a large number of intersections. Issues: 1. Not all intersections are equipped with detectors such as induction coil or video. 2. The detectors may output abnormal values which need data clean processing.
Editor's comments and enhancements are shown in green. [✓ Reviewed]
The quality of use case submissions will be evaluated for inclusion in the Working Group's Technical Report based on the application area, relevant AI technologies, credible reference sources (see References section), and the following characteristics:
[1] Data Focus & Learning: Use cases for AI system which utilizes Machine Learning, and those that use a fixed a priori knowledge base.
[2] Level of Autonomy: Use cases demonstrating several degrees (dependent, autonomous, human/critic in the loop, etc.) of AI system autonomy.
[3] Verifiability & Transparency: Use cases demonstrating several types and levels of verifiability and transparency, including approaches for explainable AI, accountability, etc.
[4] Impact: Use cases demonstrating the impact of AI systems to society, environment, etc.
[5] Architecture: Use cases demonstrating several architectural paradigms for AI systems (e.g., cloud, distributed AI, crowdsourcing, swarm intelligence, etc.)
[6] Functional aspects, trustworthiness, and societal concerns
[7] AI life cycle components include acquire/process/apply.
These characteristics are identified in red in the use case.
No.
50
ID:
Use Case Name:
AI solution to quality control of Electronic Medical Record (EMR) in real time
To insure the completeness, consistency, punctuality and medical-compliance of EMR written by physicians
Short Description (up to 150 words)
This AI solution in ET Medical Brain Medical service support system was developed that could simultaneously detect mistakes while physicians wrote EMR (Electronic Medical Record).
Using NLP (Natural Language Processing) ability, it can process a large amount of unstructured text and judge the accuracy according to recognized medical reference.
It achieved 80% coverage of all the EMR quality control requirements issued by Chinese government, and human labour of EMR QC (Quality Control) was reduced 60%, which translated into cost savings, and enhanced physician education.
Complete Description
This AI solution in ET Medical Brain Medical service support system was developed that could simultaneously detect mistakes while physicians wrote EMR (Electronic Medical Record).
Using NLP (Natural Language Processing) ability, it can process a large amount of unstructured text and judge the accuracy according to recognized medical reference.
It achieved 80% coverage of all the EMR quality control requirements issued by Chinese government, and human labour of EMR QC (Quality Control) was reduced 60%, which translated into cost savings, and enhanced physician education.
Medical records are the records of the occurrence, development and prognosis of patients' diseases, as well as the medical activities such as examination, diagnosis and treatment.
A high-quality medical record has great value at medical and legal level.
When medical records are converted from handwritten to electronic input, delayed, uncompleted writing and copying are endangering the quality of medical records.
Once the medical record data does not meet the requirements, it will greatly affect the health of patients, the development of medicine and the judgment of responsibility in medical accidents.
Nowadays, hospital has a Medical Records Department to control medical records quality manually. However, as the number of medical records increases, the inspection requirements become more complex, and the medical professional knowledge requirements are improved, so the medical records quality inspection becomes harder.
The intelligent electronic medical record quality control system is based on NLP. When a doctor writes medical records, it can analyze unstructured medical record text, and control the quality based on government requirements, ensure the integrity, consistency, timeliness and compliance of medical records.
ET (Evolutionary Technology) Medical Brain Medical service support system has learning ability to learn more medical knowledge including clinical pathway, drug compatibility taboo etc. it can learn the habits and rules of doctor’s manual review to inspects records profoundly.
The current system has covered 189 medical records quality inspection requirements, saved 60% review time for medical record department, which greatly saved the cost of the hospital, reduced the inspection time and repeated work, and will help doctors put more energy into the education and training.
Ratio of potential issues which are "of interest" for human evaluation. Ideal target is to reduce the current volume by 80%.
Improve accuracy
1
Precision
Correctly Predicted Anomalous scenarios/ Total Anomalous scenarios predicted
1
Features related to adulterants in radio spectrum
Intensities around NIR range
1
Average vehicle driving speed
Average vehicle driving speed on all the road sections in a given region
Improve the road utilization efficiency
1
Coverage
Ratio of EMR QC requirements done in the solution/all issued EMR QC requirements in China. Ideal target is 100%.
Improve accuracy
1
Accuracy
The number of correctly recognized users’ intent over total number of users. Currently, accuracy reaches 95%.
Improve accuracy of recognizing users’ intent
1
Classifier Accuracy
Without straightening and pre-processing, the average classification accuracy obtained was 68.5%. However, with preprocessing, the classification accuracy improved to 86.7%. These results are very likely to improve with more annotated training data for classification.
1
Closeness to Golden Batch
How close a process is to the best possible batch
Helps in isolation of bad batches from good batches by identifying combination of process variable trajectories that lead to good or bad batch operation.
1
Model Accuracy
Accuracy of the prediction model
The extent to which the setpoints have correctly predicted
1
Ratio of ML discovered failure rate to nominal failure rate
What combination of manufacturing processes/decisions leads to higher failure rates compared to nominal failure rate
Actionable intelligence to improve the manufacturing process of HV circuit breakers
1
Number of labors
reduced % of labors
improvement of productivity
1
Customer Satisfaction
The ratio of customer satisfaction when using this system for requests. The expectation is 100%
Increasing its ratio as high as possible
1
MIoU (Mean Intersection over Union)
The intersection of prediction area and actual area divided by the union of the predicted area and the actual area. Ideal target is 100%.
Improve accuracy
1
Classification Ratio
Real to Pseudo wrong classification
Establishes the quality of identification
1
Ease of use
Simplicity and efficiency during initial learning. Teaching process should be easy.
1
Zone of Influence/ Thermal Correlation Index
Extent of influence of ACUs on data center racks.
Helps in improved control.
1
Invisible Loss Time
Indicates the lost time of the asset in being idle or off or unplanned downtime
Asset Utilization Reports indicate the effectively utilized time there indicating the lost time and their causes
1
Prediction Accuracy
To what extent has the model been able to predict correctly
Provided ability as to % of times the quality complied
1
Algorithm accuracy
Output when compared to the human expert analysis of the same data
See Reference
1
Generation of Activities (land use information and time of travel)
Purpose of activities is assigned based on land use information and time of travel. Census data and national/ local travel surveys will provide validation for the process
Phrase 1
1
accuracy
The accuracy of infraction and incident detection from traffic pictures/videos
To increase the accuracy of traffic monitoring and inspection
2
Split
Proportion of the potential issues which are "more likely to be a valid issue" for our end users.
Average vehicle waiting time at all the intersections in a given region
Improve the road utilization efficiency
2
Resolution
The number of answers solved over total number of questions asked
Improve the resolution of questions from users
2
Annotation Completeness
35.9 chromosomes segmented out after crowd annotation, for 50 images having 46 chromosomes
2
% Reduction in Calibration Time
The amount of time saved from manually setting the calibration
2
Number of complaints
reduced % of labor's complaint
improvement of productivity
2
Accuracy
Among all the predicted customer sentiment classification, the ratio of accurate prediction, current value is 76.4%
Increasing to 90%
2
FAR (false acceptance rate)
Negative samples are identified as positive samples / Total number of negative samples.The low FAR, the more smartphone will get correct scenes and objects
Improve accuracy
2
Training efficiency
Amount of necessary data for training might lead to practical obstacles in application.
2
Overall drilling time
The time spent on one drilling job inclusive of the all downtimes
Real Time visibility into operations gives the operations early warnings to take actions immediately.
2
Generation of agents (travel times, speed on link)
Agents generated will build up in the network creating realistic conditionsw of congestion. Speed on links.
Phrase 2
2
split
Proportion of images requiring human inspection. The less the split, the higher the efficiency.
To minimize the human effort in inspection
3
Satisfaction
The number of users who are satisfied with customer service over total number of users
Improve user experience
3
Lead time
time from order to shipment
improvement of productivity
3
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 90%
Increasing to 90%
3
Initial success rate
After initial training, the success rate needs to be acceptable such that the system can be put in the production line.
3
Opeartion of service (number of users for the service)
Optimisation of route and operation time in the day. Validation provided using data collected by Mobility service operators during the operation of service
Phrase 3
3
resource utilization ratio
Achievable resource utilization ratio in the hardware infrastructure ( the higher the utilization ratio, the lower amount the required resource)
To reduce the infrastructure investment and overall solution cost
4
Accuracy
Among all the predicted customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
4
Speed of improvement
Higher convergence speed of the reinforcement algorithm is making the solution more attractive.
5
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
5
Operational efficiency
Cycle time is the primary measure in manufacturing industry.
6
Success rate
Very high success rate is required for the solution to be accepted.
Achieved 80% coverage of all the EMR quality control requirements issued by Chinese government, and human labour of EMR QC (Quality Control) was reduced 60%, which translated into cost savings, and enhanced physician education.
Editor's comments and enhancements are shown in green. [✓ Reviewed]
The quality of use case submissions will be evaluated for inclusion in the Working Group's Technical Report based on the application area, relevant AI technologies, credible reference sources (see References section), and the following characteristics:
[1] Data Focus & Learning: Use cases for AI system which utilizes Machine Learning, and those that use a fixed a priori knowledge base.
[2] Level of Autonomy: Use cases demonstrating several degrees (dependent, autonomous, human/critic in the loop, etc.) of AI system autonomy.
[3] Verifiability & Transparency: Use cases demonstrating several types and levels of verifiability and transparency, including approaches for explainable AI, accountability, etc.
[4] Impact: Use cases demonstrating the impact of AI systems to society, environment, etc.
[5] Architecture: Use cases demonstrating several architectural paradigms for AI systems (e.g., cloud, distributed AI, crowdsourcing, swarm intelligence, etc.)
[6] Functional aspects, trustworthiness, and societal concerns
[7] AI life cycle components include acquire/process/apply.
These characteristics are identified in red in the use case.
The objective is to enable flexible production resources which enable fast reconfiguration and adaptation to changing situations, context, and requirements which facilitate optimized resource usage under uncertainty.
Short Description (up to 150 words)
Rapid, and in some cases completely automated, conversion of a manufacturing facility, by changing both production capacities and production capabilities. This use case describes the adaptability of an individual factory by (physical) conversion and/or adaption of a factory’s and its machines behavior in order to adjust to changing situations like disruptions, material quality variation, production of new products, etc. A prerequisite is a modular and thereby adaptable design for manufacturing within the factory. The result is a need for intelligent and interoperable modules that basically adapted to an altered configuration on their own, and standardized interfaces between these modules.
Complete Description
Use Case description taken from [1,2,3]. Plug & Play – using a home computer and a USB cable, it is easy to connect new devices and use them almost immediately without any additional effort. The flexibility that has been available for quite a while on desktop computers is now gaining importance for industrial production. Demands on adaptability of production infrastructure are already rapidly increasing. Shorter and shorter product and innovation cycles require investment decisions for new production facilities that reflect future demand for production and process changes, where possible. In addition, the growing volatility of orders is hindering the optimal utilization of manufacturing lines with increasing frequency. Flexibility and adaptability will become increasingly important criteria in decisions regarding construction and operation of new production facilities.
One example is product labeling. Various printing technologies are available, for example tampon printers (transferring ink from the printing form to the product using an elastic tampon), inkjet printers and/or laser printers. In an adaptable factory this type of operating equipment can be connected directly to the automated production process. Simply put, the material to be printed says: “Print me”, and the tampon printer will ask: “Is the material to be printed greaseless?” The ink jet printer will then ask about the material characteristics, because it uses heat for the drying process, for example. A laser printer will ask about the material receiving the label to ensure sufficient contrast.
Use Case description taken from [1,2,3]. Plug & Play – using a home computer and a USB cable, it is easy to connect new devices and use them almost immediately without any additional effort. The flexibility that has been available for quite a while on desktop computers is now gaining importance for industrial production. Demands on adaptability of production infrastructure are already rapidly increasing. Shorter and shorter product and innovation cycles require investment decisions for new production facilities that reflect future demand for production and process changes, where possible. In addition, the growing volatility of orders is hindering the optimal utilization of manufacturing lines with increasing frequency. Flexibility and adaptability will become increasingly important criteria in decisions regarding construction and operation of new production facilities.
One example is product labeling. Various printing technologies are available, for example tampon printers (transferring ink from the printing form to the product using an elastic tampon), inkjet printers and/or laser printers. In an adaptable factory this type of operating equipment can be connected directly to the automated production process. Simply put, the material to be printed says: “Print me”, and the tampon printer will ask: “Is the material to be printed greaseless?” The ink jet printer will then ask about the material characteristics, because it uses heat for the drying process, for example. A laser printer will ask about the material receiving the label to ensure sufficient contrast.
Key aspects The application scenario for adaptable factories describes the rapid, and in some cases completely automated con-version of a manufacturing facility, by changing both production capacities and production capabilities. The key concept for implementation is a modular and thereby adaptable design for manufacturing within the factory. Intelligent and interoperable modules that basically adapted to an altered configuration on their own, and standardized interfaces between these modules allow for quick and simple conversion to adapt to changes in the market and customer demands. Whereas the application scenario Order-Controlled Production emphasizes flexible use of existing manufacturing facilities by means of intelligent connectivity, this scenario describes the adaptability of an individual factory by (physical) conversion.
Today, when creating a production line, the focus is usually not only on quality, but also maximization of productivity and profitability of a pre-conceived product range. Individual components are connected statically and are capable of producing the pre-conceived functionalities and projected volumes. Frequently, a system integrator takes care of coordinating the individual components and developing a control system for the entire facility. However, if the order level is driven by strong product individuality or high fluctuation in demand, companies can no longer rely on the advantage of particular production lines. In this case, modular, order-oriented and adaptable manufacturing configurations become more attractive: For example, they increase overall utilisation or ability to deliver products. At the same time, however, the demands on individual machines or manufacturing modules increase. Even more important than high variance of specific manufacturing steps will be the ability to combine individual modules with ease and in any situation. In order to achieve this, the modules must contain a self-description regarding their ability to be combined or converted into a machine or plant very rapidly and robustly. The following examples illustrate these requirements:
A new network-enabled field device, for example a drive with a new version of firmware, is hooked up to the production line. The new device must be provided automatically with network connectivity and be made known to all online subsystems. The participating systems must correspondingly be updated.
An unconfigured field device is introduced to production, for example to quickly replace another defective device. The field device now must be individualized and parameterized due to the information located in the software components.
A production facility is converted or modified because a new product variation is planned. The control and software related changes must be detected and automatically transmitted to all participating systems.
After conversion of a plant, it should be possible to move software components for process management around the decentralized control units, while observing certain criteria, such as output or availability.
A (new) function of the Manufacturing Execution System (MES) is inserted or altered, for example the visualization of a situation not previously required. The visualization should be done automatically and access to the necessary information from the field level should also be automatic.
This requires the mechanical engineer to design the internal development processes accordingly. Modular machines require “modular” engineering, based on libraries of re-usable modules (“platform development”). Machine architecture must be designed such that combinable mechatronic modules are created, including the Plug & Produce capability of production modules using interoperable interfaces and adaptive automation technology. This requires development of concepts for “services” across manufacturer boundaries, such as archiving, alerting or visualising, as well as a low-cost integration of MES functions.
Effect on value chains Value added is shifted from the system integrator to the machine provider or its supplier, because the machines or components are enhanced so that they are easier to integrate. The type and quality of system integration change. The present focus on (production) technology shifts to a stronger focus on organization and business processes related to production processes. In extreme cases, the system integrator could become obsolete if intelligent, self-configuring and interoperable manufacturing modules can be created at the level of the machine suppliers.
Value added for participants For manufacturing companies, a quick, inexpensive and reliable conversion of manufacturing becomes possible, so that they can react quickly to changes in customer and market demands. Increasing standardization and modularization also expand the possibilities for combining manufacturing entities of various providers and therefore realizing the most economic solution for each individual module.
Machine modularization opens up new areas with scale effects for machinery manufacturers.
Ratio of potential issues which are "of interest" for human evaluation. Ideal target is to reduce the current volume by 80%.
Improve accuracy
1
Precision
Correctly Predicted Anomalous scenarios/ Total Anomalous scenarios predicted
1
Features related to adulterants in radio spectrum
Intensities around NIR range
1
Average vehicle driving speed
Average vehicle driving speed on all the road sections in a given region
Improve the road utilization efficiency
1
Coverage
Ratio of EMR QC requirements done in the solution/all issued EMR QC requirements in China. Ideal target is 100%.
Improve accuracy
1
Accuracy
The number of correctly recognized users’ intent over total number of users. Currently, accuracy reaches 95%.
Improve accuracy of recognizing users’ intent
1
Classifier Accuracy
Without straightening and pre-processing, the average classification accuracy obtained was 68.5%. However, with preprocessing, the classification accuracy improved to 86.7%. These results are very likely to improve with more annotated training data for classification.
1
Closeness to Golden Batch
How close a process is to the best possible batch
Helps in isolation of bad batches from good batches by identifying combination of process variable trajectories that lead to good or bad batch operation.
1
Model Accuracy
Accuracy of the prediction model
The extent to which the setpoints have correctly predicted
1
Ratio of ML discovered failure rate to nominal failure rate
What combination of manufacturing processes/decisions leads to higher failure rates compared to nominal failure rate
Actionable intelligence to improve the manufacturing process of HV circuit breakers
1
Number of labors
reduced % of labors
improvement of productivity
1
Customer Satisfaction
The ratio of customer satisfaction when using this system for requests. The expectation is 100%
Increasing its ratio as high as possible
1
MIoU (Mean Intersection over Union)
The intersection of prediction area and actual area divided by the union of the predicted area and the actual area. Ideal target is 100%.
Improve accuracy
1
Classification Ratio
Real to Pseudo wrong classification
Establishes the quality of identification
1
Ease of use
Simplicity and efficiency during initial learning. Teaching process should be easy.
1
Zone of Influence/ Thermal Correlation Index
Extent of influence of ACUs on data center racks.
Helps in improved control.
1
Invisible Loss Time
Indicates the lost time of the asset in being idle or off or unplanned downtime
Asset Utilization Reports indicate the effectively utilized time there indicating the lost time and their causes
1
Prediction Accuracy
To what extent has the model been able to predict correctly
Provided ability as to % of times the quality complied
1
Algorithm accuracy
Output when compared to the human expert analysis of the same data
See Reference
1
Generation of Activities (land use information and time of travel)
Purpose of activities is assigned based on land use information and time of travel. Census data and national/ local travel surveys will provide validation for the process
Phrase 1
1
accuracy
The accuracy of infraction and incident detection from traffic pictures/videos
To increase the accuracy of traffic monitoring and inspection
2
Split
Proportion of the potential issues which are "more likely to be a valid issue" for our end users.
Average vehicle waiting time at all the intersections in a given region
Improve the road utilization efficiency
2
Resolution
The number of answers solved over total number of questions asked
Improve the resolution of questions from users
2
Annotation Completeness
35.9 chromosomes segmented out after crowd annotation, for 50 images having 46 chromosomes
2
% Reduction in Calibration Time
The amount of time saved from manually setting the calibration
2
Number of complaints
reduced % of labor's complaint
improvement of productivity
2
Accuracy
Among all the predicted customer sentiment classification, the ratio of accurate prediction, current value is 76.4%
Increasing to 90%
2
FAR (false acceptance rate)
Negative samples are identified as positive samples / Total number of negative samples.The low FAR, the more smartphone will get correct scenes and objects
Improve accuracy
2
Training efficiency
Amount of necessary data for training might lead to practical obstacles in application.
2
Overall drilling time
The time spent on one drilling job inclusive of the all downtimes
Real Time visibility into operations gives the operations early warnings to take actions immediately.
2
Generation of agents (travel times, speed on link)
Agents generated will build up in the network creating realistic conditionsw of congestion. Speed on links.
Phrase 2
2
split
Proportion of images requiring human inspection. The less the split, the higher the efficiency.
To minimize the human effort in inspection
3
Satisfaction
The number of users who are satisfied with customer service over total number of users
Improve user experience
3
Lead time
time from order to shipment
improvement of productivity
3
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 90%
Increasing to 90%
3
Initial success rate
After initial training, the success rate needs to be acceptable such that the system can be put in the production line.
3
Opeartion of service (number of users for the service)
Optimisation of route and operation time in the day. Validation provided using data collected by Mobility service operators during the operation of service
Phrase 3
3
resource utilization ratio
Achievable resource utilization ratio in the hardware infrastructure ( the higher the utilization ratio, the lower amount the required resource)
To reduce the infrastructure investment and overall solution cost
4
Accuracy
Among all the predicted customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
4
Speed of improvement
Higher convergence speed of the reinforcement algorithm is making the solution more attractive.
5
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
5
Operational efficiency
Cycle time is the primary measure in manufacturing industry.
6
Success rate
Very high success rate is required for the solution to be accepted.
AI Features
Task(s)
Automatic reasoning (e.g. [7,8]), AI (task) planning (e.g. [4,6]), distributed coordination and negotiation (e.g. [5])
Standardization needs for setting up this use case is currently under further investigation. Some initial intentions on standardization needs are the following: a vocabulary with formal semantic for symbolic reasoning about production capabilities across different vendors, standardized negotiation mechanisms, standardized autonomy classes of components, machines, etc. Quality model for trustful learned models and automatic behavior resulting from it.
Enabling flexible and autonomously reconfigurable production systems ease human-machine configuration, facilitate optimized machine use, reduce failures through autonomous compensation, optimized product quality through prediction techniques.
Editor's comments and enhancements are shown in green. [✓ Reviewed]
The quality of use case submissions will be evaluated for inclusion in the Working Group's Technical Report based on the application area, relevant AI technologies, credible reference sources (see References section), and the following characteristics:
[1] Data Focus & Learning: Use cases for AI system which utilizes Machine Learning, and those that use a fixed a priori knowledge base.
[2] Level of Autonomy: Use cases demonstrating several degrees (dependent, autonomous, human/critic in the loop, etc.) of AI system autonomy.
[3] Verifiability & Transparency: Use cases demonstrating several types and levels of verifiability and transparency, including approaches for explainable AI, accountability, etc.
[4] Impact: Use cases demonstrating the impact of AI systems to society, environment, etc.
[5] Architecture: Use cases demonstrating several architectural paradigms for AI systems (e.g., cloud, distributed AI, crowdsourcing, swarm intelligence, etc.)
[6] Functional aspects, trustworthiness, and societal concerns
[7] AI life cycle components include acquire/process/apply.
These characteristics are identified in red in the use case.
The objective is to enable automatic supplier contracting for optimized utilization of manufacturing capabilities at suppliers, novel degrees of flexibility in contract manufacturing, and enable (mass) customized customer ordering
Short Description (up to 150 words)
A network of production capabilities and capacities that extend beyond factory and company boundaries allows for a quick order-controlled adaption to changing market and order conditions. The result is a largely fragmented and dynamic value chain network that change as required by the individual order, and thereby make the best use of capabilities and capacities of existing production facilities. The goal is to allow for automated order planning, allocation and execution, thereby considering all production steps and facilities required to facilitate linking external factories into a company’s production process, as automated as possible.
Complete Description
Use Case description taken from [1,2,3]. Many contemporary products are changing at an ever-in-creasing rate. Whereas up until just recently, smartphone displays were flat, the first curved displays are already on the market. The array of materials used in the automotive sector is also continually expanding – from aluminum, to high-strength steels and even fiber-reinforced plastics, today many types of materials are used.
Innovation and product cycles are getting shorter all the time, and new production technologies are putting pressure on manufacturing companies to react more and more rapidly and make quick investment decisions regarding both consumer goods and investment goods. In order to confront this trend and avoid lengthy investment decisions, companies are starting to increase the network of their production capabilities beyond their own company boundaries.
Key aspects The Order-Controlled Production application scenario describes a flexible manufacturing configuration. Owing a network of production capabilities and capacities that extend beyond factory and company boundaries, this company can quickly adapt to a changing market and order conditions, and thereby make the best use of capabilities and capacities of existing production facilities. In this way the potential provided by a network to other factories out-side of the company’s own facilities is used to align the company’s own portfolio - and especially its production - to quickly changing customer and market demands. Specifically, manufacturing chains are optimized for various parameters, such as cost and time.
At its core, order-controlled production is based on standardization of the individual process steps on the one hand and the self-description of production facility capabilities on the other hand. This standardization allows for auto-mated order planning, allocation and execution, thereby considering all production steps and facilities required. This helps to combine individual process modules much more flexibly and earlier than previously possible, and to make use of their specific capabilities.
In this respect, companies offer their available production capacities to other companies and thereby increase the utilization of their own machinery. Other companies may access these capacities as needed, thereby temporarily expanding their own production spectrum. In so doing, available production capacities are utilized better and order fluctuations can be smoothed out. The goal is to facilitate linking external factories into a company’s production process, as automated as possible. In particular, the order placement process required for this should be executed automatically.
Effect on value chains Today’s relatively rigid and separately negotiated relation-ships between companies along the value chain will be transformed into a largely fragmented and dynamic value chain network that changes as required by the individual order. This applies both horizontally over the entire manufacturing process as well as vertically, with regard to production depth. Manufacturing companies focus on value-added steps that distinguish them significantly from other competitors. The possibility of creating fast and global client-manufacturer relationships can lead to unexpected competitive situations, because companies may change their role from order to order. Dynamically integrating production capacities will lead to better machine utilization and, as a result, diminishing demand for machinery suppliers.
Value added for participants On the one hand, manufacturing companies will be able to automatically expand their production capabilities and capacities ad hoc in line with demand, by utilizing external production modules. No investment is required. This enables companies to react very flexibly to changing market and customer demands. On the other hand, companies offering their machines on the market can optimize their utilization rates.
Ratio of potential issues which are "of interest" for human evaluation. Ideal target is to reduce the current volume by 80%.
Improve accuracy
1
Precision
Correctly Predicted Anomalous scenarios/ Total Anomalous scenarios predicted
1
Features related to adulterants in radio spectrum
Intensities around NIR range
1
Average vehicle driving speed
Average vehicle driving speed on all the road sections in a given region
Improve the road utilization efficiency
1
Coverage
Ratio of EMR QC requirements done in the solution/all issued EMR QC requirements in China. Ideal target is 100%.
Improve accuracy
1
Accuracy
The number of correctly recognized users’ intent over total number of users. Currently, accuracy reaches 95%.
Improve accuracy of recognizing users’ intent
1
Classifier Accuracy
Without straightening and pre-processing, the average classification accuracy obtained was 68.5%. However, with preprocessing, the classification accuracy improved to 86.7%. These results are very likely to improve with more annotated training data for classification.
1
Closeness to Golden Batch
How close a process is to the best possible batch
Helps in isolation of bad batches from good batches by identifying combination of process variable trajectories that lead to good or bad batch operation.
1
Model Accuracy
Accuracy of the prediction model
The extent to which the setpoints have correctly predicted
1
Ratio of ML discovered failure rate to nominal failure rate
What combination of manufacturing processes/decisions leads to higher failure rates compared to nominal failure rate
Actionable intelligence to improve the manufacturing process of HV circuit breakers
1
Number of labors
reduced % of labors
improvement of productivity
1
Customer Satisfaction
The ratio of customer satisfaction when using this system for requests. The expectation is 100%
Increasing its ratio as high as possible
1
MIoU (Mean Intersection over Union)
The intersection of prediction area and actual area divided by the union of the predicted area and the actual area. Ideal target is 100%.
Improve accuracy
1
Classification Ratio
Real to Pseudo wrong classification
Establishes the quality of identification
1
Ease of use
Simplicity and efficiency during initial learning. Teaching process should be easy.
1
Zone of Influence/ Thermal Correlation Index
Extent of influence of ACUs on data center racks.
Helps in improved control.
1
Invisible Loss Time
Indicates the lost time of the asset in being idle or off or unplanned downtime
Asset Utilization Reports indicate the effectively utilized time there indicating the lost time and their causes
1
Prediction Accuracy
To what extent has the model been able to predict correctly
Provided ability as to % of times the quality complied
1
Algorithm accuracy
Output when compared to the human expert analysis of the same data
See Reference
1
Generation of Activities (land use information and time of travel)
Purpose of activities is assigned based on land use information and time of travel. Census data and national/ local travel surveys will provide validation for the process
Phrase 1
1
accuracy
The accuracy of infraction and incident detection from traffic pictures/videos
To increase the accuracy of traffic monitoring and inspection
2
Split
Proportion of the potential issues which are "more likely to be a valid issue" for our end users.
Average vehicle waiting time at all the intersections in a given region
Improve the road utilization efficiency
2
Resolution
The number of answers solved over total number of questions asked
Improve the resolution of questions from users
2
Annotation Completeness
35.9 chromosomes segmented out after crowd annotation, for 50 images having 46 chromosomes
2
% Reduction in Calibration Time
The amount of time saved from manually setting the calibration
2
Number of complaints
reduced % of labor's complaint
improvement of productivity
2
Accuracy
Among all the predicted customer sentiment classification, the ratio of accurate prediction, current value is 76.4%
Increasing to 90%
2
FAR (false acceptance rate)
Negative samples are identified as positive samples / Total number of negative samples.The low FAR, the more smartphone will get correct scenes and objects
Improve accuracy
2
Training efficiency
Amount of necessary data for training might lead to practical obstacles in application.
2
Overall drilling time
The time spent on one drilling job inclusive of the all downtimes
Real Time visibility into operations gives the operations early warnings to take actions immediately.
2
Generation of agents (travel times, speed on link)
Agents generated will build up in the network creating realistic conditionsw of congestion. Speed on links.
Phrase 2
2
split
Proportion of images requiring human inspection. The less the split, the higher the efficiency.
To minimize the human effort in inspection
3
Satisfaction
The number of users who are satisfied with customer service over total number of users
Improve user experience
3
Lead time
time from order to shipment
improvement of productivity
3
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 90%
Increasing to 90%
3
Initial success rate
After initial training, the success rate needs to be acceptable such that the system can be put in the production line.
3
Opeartion of service (number of users for the service)
Optimisation of route and operation time in the day. Validation provided using data collected by Mobility service operators during the operation of service
Phrase 3
3
resource utilization ratio
Achievable resource utilization ratio in the hardware infrastructure ( the higher the utilization ratio, the lower amount the required resource)
To reduce the infrastructure investment and overall solution cost
4
Accuracy
Among all the predicted customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
4
Speed of improvement
Higher convergence speed of the reinforcement algorithm is making the solution more attractive.
5
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
5
Operational efficiency
Cycle time is the primary measure in manufacturing industry.
6
Success rate
Very high success rate is required for the solution to be accepted.
AI Features
Task(s)
Automatic reasoning, AI (task) planning, distributed coordination and negotiation (cf. [5-8] for details and overview)
Standardization needs for setting up this use case is currently under further investigation. Some initial intentions on standardization needs are the following: Standardization of data formats and semantic for exchanged data is enabler for this use case where multiple companies and institutions are involved (formal semantics for reasoning about 3d models, task decomposition and planning), standardization of interaction protocols between participants (esp. coordination and negotiation) enables automatic cross-company contracting.
Editor's comments and enhancements are shown in green. [✓ Reviewed]
The quality of use case submissions will be evaluated for inclusion in the Working Group's Technical Report based on the application area, relevant AI technologies, credible reference sources (see References section), and the following characteristics:
[1] Data Focus & Learning: Use cases for AI system which utilizes Machine Learning, and those that use a fixed a priori knowledge base.
[2] Level of Autonomy: Use cases demonstrating several degrees (dependent, autonomous, human/critic in the loop, etc.) of AI system autonomy.
[3] Verifiability & Transparency: Use cases demonstrating several types and levels of verifiability and transparency, including approaches for explainable AI, accountability, etc.
[4] Impact: Use cases demonstrating the impact of AI systems to society, environment, etc.
[5] Architecture: Use cases demonstrating several architectural paradigms for AI systems (e.g., cloud, distributed AI, crowdsourcing, swarm intelligence, etc.)
[6] Functional aspects, trustworthiness, and societal concerns
[7] AI life cycle components include acquire/process/apply.
These characteristics are identified in red in the use case.
To recognize and understand users’ intent by AI and deep learning technologies and apply such technologies to build chat bot systems to further reduce labor cost and to be applied in various fields.
Short Description (up to 150 words)
Intelligent customer service chat bot is mainly used to categorize users’ questions, recognize users’ intents and answer users’ questions intelligently for different business jobs. Currently, this chat bot has been used to handle 90% of online customer service and has enabled JD.com to save over 100 million labor costs every year.
Complete Description
JD.com has been committed to using technology to drive business growth and improve user experience in all customer service fields. Based on the improvement of customer consulting experience and the developing trend of artificial intelligence technology, as early as 2012, JD had decided to develop intelligent chat bots to fulfill the needs of continuous expansion of business, to save customer service costs and increase service capability. Intent recognition is a key and core technology to build such an intelligent customer service chat bot. By applying natural language processing technologies, deep learning technologies, traditional machine learning algorithms, intent recognition accuracy has reached to 95%. Based on accurate intents, and a series of solution finding algorithms, our chat bot can solve the user’s problems to a great extent and give the user a high quality consulting experience. Finally, in order to provide diversified and personalized customer services, we are continuously improving the accuracy of intent recognition, personalized solution generation, sentiment recognition, and image recognition. So far, intelligent customer service has revolutionized the traditional customer service consulting business.
high semantic ambiguity, Multiple language expressions in one sentence
Performance Indicators (KPIs)
Seq. No.
Name
Description
Reference to mentioned use case objectives
1
Coverage
Ratio of potential issues which are "of interest" for human evaluation. Ideal target is to reduce the current volume by 80%.
Improve accuracy
1
Precision
Correctly Predicted Anomalous scenarios/ Total Anomalous scenarios predicted
1
Features related to adulterants in radio spectrum
Intensities around NIR range
1
Average vehicle driving speed
Average vehicle driving speed on all the road sections in a given region
Improve the road utilization efficiency
1
Coverage
Ratio of EMR QC requirements done in the solution/all issued EMR QC requirements in China. Ideal target is 100%.
Improve accuracy
1
Accuracy
The number of correctly recognized users’ intent over total number of users. Currently, accuracy reaches 95%.
Improve accuracy of recognizing users’ intent
1
Classifier Accuracy
Without straightening and pre-processing, the average classification accuracy obtained was 68.5%. However, with preprocessing, the classification accuracy improved to 86.7%. These results are very likely to improve with more annotated training data for classification.
1
Closeness to Golden Batch
How close a process is to the best possible batch
Helps in isolation of bad batches from good batches by identifying combination of process variable trajectories that lead to good or bad batch operation.
1
Model Accuracy
Accuracy of the prediction model
The extent to which the setpoints have correctly predicted
1
Ratio of ML discovered failure rate to nominal failure rate
What combination of manufacturing processes/decisions leads to higher failure rates compared to nominal failure rate
Actionable intelligence to improve the manufacturing process of HV circuit breakers
1
Number of labors
reduced % of labors
improvement of productivity
1
Customer Satisfaction
The ratio of customer satisfaction when using this system for requests. The expectation is 100%
Increasing its ratio as high as possible
1
MIoU (Mean Intersection over Union)
The intersection of prediction area and actual area divided by the union of the predicted area and the actual area. Ideal target is 100%.
Improve accuracy
1
Classification Ratio
Real to Pseudo wrong classification
Establishes the quality of identification
1
Ease of use
Simplicity and efficiency during initial learning. Teaching process should be easy.
1
Zone of Influence/ Thermal Correlation Index
Extent of influence of ACUs on data center racks.
Helps in improved control.
1
Invisible Loss Time
Indicates the lost time of the asset in being idle or off or unplanned downtime
Asset Utilization Reports indicate the effectively utilized time there indicating the lost time and their causes
1
Prediction Accuracy
To what extent has the model been able to predict correctly
Provided ability as to % of times the quality complied
1
Algorithm accuracy
Output when compared to the human expert analysis of the same data
See Reference
1
Generation of Activities (land use information and time of travel)
Purpose of activities is assigned based on land use information and time of travel. Census data and national/ local travel surveys will provide validation for the process
Phrase 1
1
accuracy
The accuracy of infraction and incident detection from traffic pictures/videos
To increase the accuracy of traffic monitoring and inspection
2
Split
Proportion of the potential issues which are "more likely to be a valid issue" for our end users.
Average vehicle waiting time at all the intersections in a given region
Improve the road utilization efficiency
2
Resolution
The number of answers solved over total number of questions asked
Improve the resolution of questions from users
2
Annotation Completeness
35.9 chromosomes segmented out after crowd annotation, for 50 images having 46 chromosomes
2
% Reduction in Calibration Time
The amount of time saved from manually setting the calibration
2
Number of complaints
reduced % of labor's complaint
improvement of productivity
2
Accuracy
Among all the predicted customer sentiment classification, the ratio of accurate prediction, current value is 76.4%
Increasing to 90%
2
FAR (false acceptance rate)
Negative samples are identified as positive samples / Total number of negative samples.The low FAR, the more smartphone will get correct scenes and objects
Improve accuracy
2
Training efficiency
Amount of necessary data for training might lead to practical obstacles in application.
2
Overall drilling time
The time spent on one drilling job inclusive of the all downtimes
Real Time visibility into operations gives the operations early warnings to take actions immediately.
2
Generation of agents (travel times, speed on link)
Agents generated will build up in the network creating realistic conditionsw of congestion. Speed on links.
Phrase 2
2
split
Proportion of images requiring human inspection. The less the split, the higher the efficiency.
To minimize the human effort in inspection
3
Satisfaction
The number of users who are satisfied with customer service over total number of users
Improve user experience
3
Lead time
time from order to shipment
improvement of productivity
3
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 90%
Increasing to 90%
3
Initial success rate
After initial training, the success rate needs to be acceptable such that the system can be put in the production line.
3
Opeartion of service (number of users for the service)
Optimisation of route and operation time in the day. Validation provided using data collected by Mobility service operators during the operation of service
Phrase 3
3
resource utilization ratio
Achievable resource utilization ratio in the hardware infrastructure ( the higher the utilization ratio, the lower amount the required resource)
To reduce the infrastructure investment and overall solution cost
4
Accuracy
Among all the predicted customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
4
Speed of improvement
Higher convergence speed of the reinforcement algorithm is making the solution more attractive.
5
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
5
Operational efficiency
Cycle time is the primary measure in manufacturing industry.
6
Success rate
Very high success rate is required for the solution to be accepted.
Editor's comments and enhancements are shown in green. [✓ Reviewed]
The quality of use case submissions will be evaluated for inclusion in the Working Group's Technical Report based on the application area, relevant AI technologies, credible reference sources (see References section), and the following characteristics:
[1] Data Focus & Learning: Use cases for AI system which utilizes Machine Learning, and those that use a fixed a priori knowledge base.
[2] Level of Autonomy: Use cases demonstrating several degrees (dependent, autonomous, human/critic in the loop, etc.) of AI system autonomy.
[3] Verifiability & Transparency: Use cases demonstrating several types and levels of verifiability and transparency, including approaches for explainable AI, accountability, etc.
[4] Impact: Use cases demonstrating the impact of AI systems to society, environment, etc.
[5] Architecture: Use cases demonstrating several architectural paradigms for AI systems (e.g., cloud, distributed AI, crowdsourcing, swarm intelligence, etc.)
[6] Functional aspects, trustworthiness, and societal concerns
[7] AI life cycle components include acquire/process/apply.
These characteristics are identified in red in the use case.
Automating Karyotyping of the chromosomes in cell spread images.
Segmentation of chromosomes in the images using non expert crowd
Short Description (up to 150 words)
Karyotyping of the chromosomes micro-photographed under metaphase is done by characterizing the individual chromosomes in cell spread images. Currently, considerable effort and time is spent to manually segment out chromosomes from cell images, and classifying the segmented chromosomes. We proposed a method to segment out and classify chromosomes for healthy patients using a combination of crowdsourcing, preprocessing and deep learning, wherein the non-expert crowd from external crowdsourcing platform is utilized to segment out the chromosomes, which are then classified using deep neural network. Results are encouraging and promise to significantly reduce the cognitive burden of segmenting and karyotyping chromosomes.
Complete Description
Metaphase chromosome analysis is one of the primary techniques utilized in cytogenetics. Observations of chromosomal segments or translocations during metaphase can indicate structural changes in the cell genome, and is often used for diagnostic purposes. Karyotyping of the chromosomes micro-photographed under metaphase is done by characterizing the individual chromosomes in cell spread images. Currently, considerable effort and time is spent to manually segment out chromosomes from cell images, and classifying the segmented chromosomes into one of the 24 types, or for diseased cells to one of the known translocated types. Segmenting out the chromosomes in such images can be especially laborious and is often done manually, if there are overlapping chromosomes in the image which are not easily separable by image processing techniques. Many techniques have been proposed to automate the segmentation and classification of chromosomes from spread images with reasonable accuracy, but given the criticality of the domain, a human in the loop is often still required. In this paper, we present a method to segment out and classify chromosomes for healthy patients using a combination of crowdsourcing, preprocessing and deep learning, wherein the non-expert crowd from CrowdFlower is utilized to segment out the chromosomes from the cell image, which are then straightened and fed into a (hierarchical) deep neural network for classification. Experiments are performed on 400 real healthy patient images obtained from a hospital. Results are encouraging and promise to significantly reduce the cognitive burden of segmenting and karyotyping chromosomes.
Ratio of potential issues which are "of interest" for human evaluation. Ideal target is to reduce the current volume by 80%.
Improve accuracy
1
Precision
Correctly Predicted Anomalous scenarios/ Total Anomalous scenarios predicted
1
Features related to adulterants in radio spectrum
Intensities around NIR range
1
Average vehicle driving speed
Average vehicle driving speed on all the road sections in a given region
Improve the road utilization efficiency
1
Coverage
Ratio of EMR QC requirements done in the solution/all issued EMR QC requirements in China. Ideal target is 100%.
Improve accuracy
1
Accuracy
The number of correctly recognized users’ intent over total number of users. Currently, accuracy reaches 95%.
Improve accuracy of recognizing users’ intent
1
Classifier Accuracy
Without straightening and pre-processing, the average classification accuracy obtained was 68.5%. However, with preprocessing, the classification accuracy improved to 86.7%. These results are very likely to improve with more annotated training data for classification.
1
Closeness to Golden Batch
How close a process is to the best possible batch
Helps in isolation of bad batches from good batches by identifying combination of process variable trajectories that lead to good or bad batch operation.
1
Model Accuracy
Accuracy of the prediction model
The extent to which the setpoints have correctly predicted
1
Ratio of ML discovered failure rate to nominal failure rate
What combination of manufacturing processes/decisions leads to higher failure rates compared to nominal failure rate
Actionable intelligence to improve the manufacturing process of HV circuit breakers
1
Number of labors
reduced % of labors
improvement of productivity
1
Customer Satisfaction
The ratio of customer satisfaction when using this system for requests. The expectation is 100%
Increasing its ratio as high as possible
1
MIoU (Mean Intersection over Union)
The intersection of prediction area and actual area divided by the union of the predicted area and the actual area. Ideal target is 100%.
Improve accuracy
1
Classification Ratio
Real to Pseudo wrong classification
Establishes the quality of identification
1
Ease of use
Simplicity and efficiency during initial learning. Teaching process should be easy.
1
Zone of Influence/ Thermal Correlation Index
Extent of influence of ACUs on data center racks.
Helps in improved control.
1
Invisible Loss Time
Indicates the lost time of the asset in being idle or off or unplanned downtime
Asset Utilization Reports indicate the effectively utilized time there indicating the lost time and their causes
1
Prediction Accuracy
To what extent has the model been able to predict correctly
Provided ability as to % of times the quality complied
1
Algorithm accuracy
Output when compared to the human expert analysis of the same data
See Reference
1
Generation of Activities (land use information and time of travel)
Purpose of activities is assigned based on land use information and time of travel. Census data and national/ local travel surveys will provide validation for the process
Phrase 1
1
accuracy
The accuracy of infraction and incident detection from traffic pictures/videos
To increase the accuracy of traffic monitoring and inspection
2
Split
Proportion of the potential issues which are "more likely to be a valid issue" for our end users.
Average vehicle waiting time at all the intersections in a given region
Improve the road utilization efficiency
2
Resolution
The number of answers solved over total number of questions asked
Improve the resolution of questions from users
2
Annotation Completeness
35.9 chromosomes segmented out after crowd annotation, for 50 images having 46 chromosomes
2
% Reduction in Calibration Time
The amount of time saved from manually setting the calibration
2
Number of complaints
reduced % of labor's complaint
improvement of productivity
2
Accuracy
Among all the predicted customer sentiment classification, the ratio of accurate prediction, current value is 76.4%
Increasing to 90%
2
FAR (false acceptance rate)
Negative samples are identified as positive samples / Total number of negative samples.The low FAR, the more smartphone will get correct scenes and objects
Improve accuracy
2
Training efficiency
Amount of necessary data for training might lead to practical obstacles in application.
2
Overall drilling time
The time spent on one drilling job inclusive of the all downtimes
Real Time visibility into operations gives the operations early warnings to take actions immediately.
2
Generation of agents (travel times, speed on link)
Agents generated will build up in the network creating realistic conditionsw of congestion. Speed on links.
Phrase 2
2
split
Proportion of images requiring human inspection. The less the split, the higher the efficiency.
To minimize the human effort in inspection
3
Satisfaction
The number of users who are satisfied with customer service over total number of users
Improve user experience
3
Lead time
time from order to shipment
improvement of productivity
3
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 90%
Increasing to 90%
3
Initial success rate
After initial training, the success rate needs to be acceptable such that the system can be put in the production line.
3
Opeartion of service (number of users for the service)
Optimisation of route and operation time in the day. Validation provided using data collected by Mobility service operators during the operation of service
Phrase 3
3
resource utilization ratio
Achievable resource utilization ratio in the hardware infrastructure ( the higher the utilization ratio, the lower amount the required resource)
To reduce the infrastructure investment and overall solution cost
4
Accuracy
Among all the predicted customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
4
Speed of improvement
Higher convergence speed of the reinforcement algorithm is making the solution more attractive.
5
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
5
Operational efficiency
Cycle time is the primary measure in manufacturing industry.
6
Success rate
Very high success rate is required for the solution to be accepted.
The dataset comprised of 400 stained images with varying degrees of overlap between chromosomes, out of which 200 were kept for testing and the remaining for training and validation
Editor's comments and enhancements are shown in green. [✓ Reviewed]
The quality of use case submissions will be evaluated for inclusion in the Working Group's Technical Report based on the application area, relevant AI technologies, credible reference sources (see References section), and the following characteristics:
[1] Data Focus & Learning: Use cases for AI system which utilizes Machine Learning, and those that use a fixed a priori knowledge base.
[2] Level of Autonomy: Use cases demonstrating several degrees (dependent, autonomous, human/critic in the loop, etc.) of AI system autonomy.
[3] Verifiability & Transparency: Use cases demonstrating several types and levels of verifiability and transparency, including approaches for explainable AI, accountability, etc.
[4] Impact: Use cases demonstrating the impact of AI systems to society, environment, etc.
[5] Architecture: Use cases demonstrating several architectural paradigms for AI systems (e.g., cloud, distributed AI, crowdsourcing, swarm intelligence, etc.)
[6] Functional aspects, trustworthiness, and societal concerns
[7] AI life cycle components include acquire/process/apply.
These characteristics are identified in red in the use case.
No.
38
ID:
Use Case Name:
Machine learning driven approach to identify the weak spots in the manufacturing of the circuit breakers.
Detecting the issues in manufacturing process that leads to early failures of the circuit breakers through the data mining of the manufacturing process.
To generate actionable intelligence to improve the manufacturing process of circuit breakers through mining of manufacturing related data.
Short Description (up to 150 words)
An approach was developed that can mine the manufacturing data of circuit breakers through multiple machine learning algorithms. The approach could successfully identify the weak spots in the manufacturing where failure rate jumped from 0.2% to 7% (35 fold more probability of failure) and hence candidates for improvement in the manufacturing process.
Complete Description
High voltage circuit breakers are critical component of an electric circuit and it has a normal lifespan of 30-40 years. However, due to various reasons few circuit breakers fail within 0-5 years of operation. As a manufacturer of these circuit breakers, lots of data related to manufacturing aspects are present with the manufacturer. Such data has information about production lot size, material of production, design voltages for sub-components, heater voltages, date of failure etc. In general data related to 49 variables are captured for close to 56000 circuit breakers over a lifespan of several years. The manufacturer is interested to know if there are any weak spots in the manufacturing process which leads to higher failure rates. Circuit breakers can fail not only due to manufacturing defects but also due to wrong operation of the circuit breaker in the field e.g. applying voltages higher than design values. However, operational data of the circuit breakers was not available with the manufacturer. Therefore, the key challenge of this project was knowledge discovery with partial data set using machine learning algorithms. The data scientists applied various machine learning algorithms such as decision tree, random forest, support vector machine, Naïve Bayes classifier, logistic regression and neural network and compared the results of one algorithm verses the other algorithm. Through multiple numerical experimentations on data selection and algorithm hyper parameter tuning, the data scientist team selected the best algorithms and deduced the key weak spots in the manufacturing that are generally associated with high failure rates. In conclusion, the work provided a set of 5 actionable rules, where the failure rates jumped drastically from 0.2% to 7% leading to 35-fold higher chance of failure.
Ratio of potential issues which are "of interest" for human evaluation. Ideal target is to reduce the current volume by 80%.
Improve accuracy
1
Precision
Correctly Predicted Anomalous scenarios/ Total Anomalous scenarios predicted
1
Features related to adulterants in radio spectrum
Intensities around NIR range
1
Average vehicle driving speed
Average vehicle driving speed on all the road sections in a given region
Improve the road utilization efficiency
1
Coverage
Ratio of EMR QC requirements done in the solution/all issued EMR QC requirements in China. Ideal target is 100%.
Improve accuracy
1
Accuracy
The number of correctly recognized users’ intent over total number of users. Currently, accuracy reaches 95%.
Improve accuracy of recognizing users’ intent
1
Classifier Accuracy
Without straightening and pre-processing, the average classification accuracy obtained was 68.5%. However, with preprocessing, the classification accuracy improved to 86.7%. These results are very likely to improve with more annotated training data for classification.
1
Closeness to Golden Batch
How close a process is to the best possible batch
Helps in isolation of bad batches from good batches by identifying combination of process variable trajectories that lead to good or bad batch operation.
1
Model Accuracy
Accuracy of the prediction model
The extent to which the setpoints have correctly predicted
1
Ratio of ML discovered failure rate to nominal failure rate
What combination of manufacturing processes/decisions leads to higher failure rates compared to nominal failure rate
Actionable intelligence to improve the manufacturing process of HV circuit breakers
1
Number of labors
reduced % of labors
improvement of productivity
1
Customer Satisfaction
The ratio of customer satisfaction when using this system for requests. The expectation is 100%
Increasing its ratio as high as possible
1
MIoU (Mean Intersection over Union)
The intersection of prediction area and actual area divided by the union of the predicted area and the actual area. Ideal target is 100%.
Improve accuracy
1
Classification Ratio
Real to Pseudo wrong classification
Establishes the quality of identification
1
Ease of use
Simplicity and efficiency during initial learning. Teaching process should be easy.
1
Zone of Influence/ Thermal Correlation Index
Extent of influence of ACUs on data center racks.
Helps in improved control.
1
Invisible Loss Time
Indicates the lost time of the asset in being idle or off or unplanned downtime
Asset Utilization Reports indicate the effectively utilized time there indicating the lost time and their causes
1
Prediction Accuracy
To what extent has the model been able to predict correctly
Provided ability as to % of times the quality complied
1
Algorithm accuracy
Output when compared to the human expert analysis of the same data
See Reference
1
Generation of Activities (land use information and time of travel)
Purpose of activities is assigned based on land use information and time of travel. Census data and national/ local travel surveys will provide validation for the process
Phrase 1
1
accuracy
The accuracy of infraction and incident detection from traffic pictures/videos
To increase the accuracy of traffic monitoring and inspection
2
Split
Proportion of the potential issues which are "more likely to be a valid issue" for our end users.
Average vehicle waiting time at all the intersections in a given region
Improve the road utilization efficiency
2
Resolution
The number of answers solved over total number of questions asked
Improve the resolution of questions from users
2
Annotation Completeness
35.9 chromosomes segmented out after crowd annotation, for 50 images having 46 chromosomes
2
% Reduction in Calibration Time
The amount of time saved from manually setting the calibration
2
Number of complaints
reduced % of labor's complaint
improvement of productivity
2
Accuracy
Among all the predicted customer sentiment classification, the ratio of accurate prediction, current value is 76.4%
Increasing to 90%
2
FAR (false acceptance rate)
Negative samples are identified as positive samples / Total number of negative samples.The low FAR, the more smartphone will get correct scenes and objects
Improve accuracy
2
Training efficiency
Amount of necessary data for training might lead to practical obstacles in application.
2
Overall drilling time
The time spent on one drilling job inclusive of the all downtimes
Real Time visibility into operations gives the operations early warnings to take actions immediately.
2
Generation of agents (travel times, speed on link)
Agents generated will build up in the network creating realistic conditionsw of congestion. Speed on links.
Phrase 2
2
split
Proportion of images requiring human inspection. The less the split, the higher the efficiency.
To minimize the human effort in inspection
3
Satisfaction
The number of users who are satisfied with customer service over total number of users
Improve user experience
3
Lead time
time from order to shipment
improvement of productivity
3
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 90%
Increasing to 90%
3
Initial success rate
After initial training, the success rate needs to be acceptable such that the system can be put in the production line.
3
Opeartion of service (number of users for the service)
Optimisation of route and operation time in the day. Validation provided using data collected by Mobility service operators during the operation of service
Phrase 3
3
resource utilization ratio
Achievable resource utilization ratio in the hardware infrastructure ( the higher the utilization ratio, the lower amount the required resource)
To reduce the infrastructure investment and overall solution cost
4
Accuracy
Among all the predicted customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
4
Speed of improvement
Higher convergence speed of the reinforcement algorithm is making the solution more attractive.
5
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
5
Operational efficiency
Cycle time is the primary measure in manufacturing industry.
6
Success rate
Very high success rate is required for the solution to be accepted.
Editor's comments and enhancements are shown in green. [✓ Reviewed]
The quality of use case submissions will be evaluated for inclusion in the Working Group's Technical Report based on the application area, relevant AI technologies, credible reference sources (see References section), and the following characteristics:
[1] Data Focus & Learning: Use cases for AI system which utilizes Machine Learning, and those that use a fixed a priori knowledge base.
[2] Level of Autonomy: Use cases demonstrating several degrees (dependent, autonomous, human/critic in the loop, etc.) of AI system autonomy.
[3] Verifiability & Transparency: Use cases demonstrating several types and levels of verifiability and transparency, including approaches for explainable AI, accountability, etc.
[4] Impact: Use cases demonstrating the impact of AI systems to society, environment, etc.
[5] Architecture: Use cases demonstrating several architectural paradigms for AI systems (e.g., cloud, distributed AI, crowdsourcing, swarm intelligence, etc.)
[6] Functional aspects, trustworthiness, and societal concerns
[7] AI life cycle components include acquire/process/apply.
These characteristics are identified in red in the use case.
No.
39
ID:
Use Case Name:
Machine Learning Driven Analysis of Batch Process Operation Data to Identify Causes for Poor Batch Performance
Provide insight to the operation team to improve the productivity of batch manufacturing through machine learning on historical operation data
Short Description (up to 150 words)
An approach was developed that can use machine learning models to identify issues in batch manufacturing.
Complete Description
Batch operation is generally quite complex involving dynamics in the operation and interplay of various process variables. Due to this, sometimes, few batches end up running slower than nominal batch time and few batches also yield bad quality end products resulting in significant production loss. Additionally, often in the industrial context, data size and variety are limited and to develop a robust machine learning model from limited available data sets is a challenging task. Due to transient nature of batch operation data, the traditional PCA algorithm fails in analyzing the batch data and hence MPCA was applied as logical extension of PCA algorithm. As MPCA naturally considers the dynamics in the data and inter-correlations among the process variables, it provides a valuable insight on the batch data. The approach was successfully demonstrated on milk pasteurization process data where only 4 batches were provided for modelling. Using such 4 seed batches, the algorithm synthetically creates 50 batches of data and introduction of anomalies in some batches. Concept of design of experiments and stochastic perturbations are used in synthetic generation of the data set. The work was able to successfully build a robust MPCA model with such data and isolate the bad batches of data from good batches of the data. Additionally, through contribution plots, the algorithm identifies when a certain batch drifted from nominal operation and which variables are the root causes for the bad batch operation.
Ratio of potential issues which are "of interest" for human evaluation. Ideal target is to reduce the current volume by 80%.
Improve accuracy
1
Precision
Correctly Predicted Anomalous scenarios/ Total Anomalous scenarios predicted
1
Features related to adulterants in radio spectrum
Intensities around NIR range
1
Average vehicle driving speed
Average vehicle driving speed on all the road sections in a given region
Improve the road utilization efficiency
1
Coverage
Ratio of EMR QC requirements done in the solution/all issued EMR QC requirements in China. Ideal target is 100%.
Improve accuracy
1
Accuracy
The number of correctly recognized users’ intent over total number of users. Currently, accuracy reaches 95%.
Improve accuracy of recognizing users’ intent
1
Classifier Accuracy
Without straightening and pre-processing, the average classification accuracy obtained was 68.5%. However, with preprocessing, the classification accuracy improved to 86.7%. These results are very likely to improve with more annotated training data for classification.
1
Closeness to Golden Batch
How close a process is to the best possible batch
Helps in isolation of bad batches from good batches by identifying combination of process variable trajectories that lead to good or bad batch operation.
1
Model Accuracy
Accuracy of the prediction model
The extent to which the setpoints have correctly predicted
1
Ratio of ML discovered failure rate to nominal failure rate
What combination of manufacturing processes/decisions leads to higher failure rates compared to nominal failure rate
Actionable intelligence to improve the manufacturing process of HV circuit breakers
1
Number of labors
reduced % of labors
improvement of productivity
1
Customer Satisfaction
The ratio of customer satisfaction when using this system for requests. The expectation is 100%
Increasing its ratio as high as possible
1
MIoU (Mean Intersection over Union)
The intersection of prediction area and actual area divided by the union of the predicted area and the actual area. Ideal target is 100%.
Improve accuracy
1
Classification Ratio
Real to Pseudo wrong classification
Establishes the quality of identification
1
Ease of use
Simplicity and efficiency during initial learning. Teaching process should be easy.
1
Zone of Influence/ Thermal Correlation Index
Extent of influence of ACUs on data center racks.
Helps in improved control.
1
Invisible Loss Time
Indicates the lost time of the asset in being idle or off or unplanned downtime
Asset Utilization Reports indicate the effectively utilized time there indicating the lost time and their causes
1
Prediction Accuracy
To what extent has the model been able to predict correctly
Provided ability as to % of times the quality complied
1
Algorithm accuracy
Output when compared to the human expert analysis of the same data
See Reference
1
Generation of Activities (land use information and time of travel)
Purpose of activities is assigned based on land use information and time of travel. Census data and national/ local travel surveys will provide validation for the process
Phrase 1
1
accuracy
The accuracy of infraction and incident detection from traffic pictures/videos
To increase the accuracy of traffic monitoring and inspection
2
Split
Proportion of the potential issues which are "more likely to be a valid issue" for our end users.
Average vehicle waiting time at all the intersections in a given region
Improve the road utilization efficiency
2
Resolution
The number of answers solved over total number of questions asked
Improve the resolution of questions from users
2
Annotation Completeness
35.9 chromosomes segmented out after crowd annotation, for 50 images having 46 chromosomes
2
% Reduction in Calibration Time
The amount of time saved from manually setting the calibration
2
Number of complaints
reduced % of labor's complaint
improvement of productivity
2
Accuracy
Among all the predicted customer sentiment classification, the ratio of accurate prediction, current value is 76.4%
Increasing to 90%
2
FAR (false acceptance rate)
Negative samples are identified as positive samples / Total number of negative samples.The low FAR, the more smartphone will get correct scenes and objects
Improve accuracy
2
Training efficiency
Amount of necessary data for training might lead to practical obstacles in application.
2
Overall drilling time
The time spent on one drilling job inclusive of the all downtimes
Real Time visibility into operations gives the operations early warnings to take actions immediately.
2
Generation of agents (travel times, speed on link)
Agents generated will build up in the network creating realistic conditionsw of congestion. Speed on links.
Phrase 2
2
split
Proportion of images requiring human inspection. The less the split, the higher the efficiency.
To minimize the human effort in inspection
3
Satisfaction
The number of users who are satisfied with customer service over total number of users
Improve user experience
3
Lead time
time from order to shipment
improvement of productivity
3
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 90%
Increasing to 90%
3
Initial success rate
After initial training, the success rate needs to be acceptable such that the system can be put in the production line.
3
Opeartion of service (number of users for the service)
Optimisation of route and operation time in the day. Validation provided using data collected by Mobility service operators during the operation of service
Phrase 3
3
resource utilization ratio
Achievable resource utilization ratio in the hardware infrastructure ( the higher the utilization ratio, the lower amount the required resource)
To reduce the infrastructure investment and overall solution cost
4
Accuracy
Among all the predicted customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
4
Speed of improvement
Higher convergence speed of the reinforcement algorithm is making the solution more attractive.
5
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
5
Operational efficiency
Cycle time is the primary measure in manufacturing industry.
6
Success rate
Very high success rate is required for the solution to be accepted.
Editor's comments and enhancements are shown in green. [✓ Reviewed]
The quality of use case submissions will be evaluated for inclusion in the Working Group's Technical Report based on the application area, relevant AI technologies, credible reference sources (see References section), and the following characteristics:
[1] Data Focus & Learning: Use cases for AI system which utilizes Machine Learning, and those that use a fixed a priori knowledge base.
[2] Level of Autonomy: Use cases demonstrating several degrees (dependent, autonomous, human/critic in the loop, etc.) of AI system autonomy.
[3] Verifiability & Transparency: Use cases demonstrating several types and levels of verifiability and transparency, including approaches for explainable AI, accountability, etc.
[4] Impact: Use cases demonstrating the impact of AI systems to society, environment, etc.
[5] Architecture: Use cases demonstrating several architectural paradigms for AI systems (e.g., cloud, distributed AI, crowdsourcing, swarm intelligence, etc.)
[6] Functional aspects, trustworthiness, and societal concerns
[7] AI life cycle components include acquire/process/apply.
These characteristics are identified in red in the use case.
No.
40
ID:
Use Case Name:
Empowering Autonomous Flow meter control- Reducing time taken to -proving of meters-
Reduce the time taken for trial & error methods to set the VFD and FCV setpoints
Short Description (up to 150 words)
The customer had to set VFD and FCV % manually to achieve desired flowrate using trial & error methods, which could take about 3-4 hours. Efficiency for the proving of the meters was very less & improvement was needed to remove any aberration in reading as it was time consuming.
Complete Description
Cerebra was integrated with the system considering the flow of the fluid. The customer can choose between the available options of high flow rate, low flow rate or multi viscous flow. Then, with the master meter in the loop of testing, the meter from the field was introduced to analyse how much of aberration is there and then proving it more efficiently. Since it took more time for them to get the exact values of VFD & FCV % to achieve the desired flow rate, Cerebra’s Prognostics Engine was introduced. Purely based upon machine learning algorithms, the data models for the VFD & FCV % was used to predict the values to be chosen with an accuracy of about 98%. Since there was a presence of a closed-loop system, this predicted value was automatically registered on the valves’ monitors which only required small tweaking in the end, thus reduced human efforts.
Ratio of potential issues which are "of interest" for human evaluation. Ideal target is to reduce the current volume by 80%.
Improve accuracy
1
Precision
Correctly Predicted Anomalous scenarios/ Total Anomalous scenarios predicted
1
Features related to adulterants in radio spectrum
Intensities around NIR range
1
Average vehicle driving speed
Average vehicle driving speed on all the road sections in a given region
Improve the road utilization efficiency
1
Coverage
Ratio of EMR QC requirements done in the solution/all issued EMR QC requirements in China. Ideal target is 100%.
Improve accuracy
1
Accuracy
The number of correctly recognized users’ intent over total number of users. Currently, accuracy reaches 95%.
Improve accuracy of recognizing users’ intent
1
Classifier Accuracy
Without straightening and pre-processing, the average classification accuracy obtained was 68.5%. However, with preprocessing, the classification accuracy improved to 86.7%. These results are very likely to improve with more annotated training data for classification.
1
Closeness to Golden Batch
How close a process is to the best possible batch
Helps in isolation of bad batches from good batches by identifying combination of process variable trajectories that lead to good or bad batch operation.
1
Model Accuracy
Accuracy of the prediction model
The extent to which the setpoints have correctly predicted
1
Ratio of ML discovered failure rate to nominal failure rate
What combination of manufacturing processes/decisions leads to higher failure rates compared to nominal failure rate
Actionable intelligence to improve the manufacturing process of HV circuit breakers
1
Number of labors
reduced % of labors
improvement of productivity
1
Customer Satisfaction
The ratio of customer satisfaction when using this system for requests. The expectation is 100%
Increasing its ratio as high as possible
1
MIoU (Mean Intersection over Union)
The intersection of prediction area and actual area divided by the union of the predicted area and the actual area. Ideal target is 100%.
Improve accuracy
1
Classification Ratio
Real to Pseudo wrong classification
Establishes the quality of identification
1
Ease of use
Simplicity and efficiency during initial learning. Teaching process should be easy.
1
Zone of Influence/ Thermal Correlation Index
Extent of influence of ACUs on data center racks.
Helps in improved control.
1
Invisible Loss Time
Indicates the lost time of the asset in being idle or off or unplanned downtime
Asset Utilization Reports indicate the effectively utilized time there indicating the lost time and their causes
1
Prediction Accuracy
To what extent has the model been able to predict correctly
Provided ability as to % of times the quality complied
1
Algorithm accuracy
Output when compared to the human expert analysis of the same data
See Reference
1
Generation of Activities (land use information and time of travel)
Purpose of activities is assigned based on land use information and time of travel. Census data and national/ local travel surveys will provide validation for the process
Phrase 1
1
accuracy
The accuracy of infraction and incident detection from traffic pictures/videos
To increase the accuracy of traffic monitoring and inspection
2
Split
Proportion of the potential issues which are "more likely to be a valid issue" for our end users.
Average vehicle waiting time at all the intersections in a given region
Improve the road utilization efficiency
2
Resolution
The number of answers solved over total number of questions asked
Improve the resolution of questions from users
2
Annotation Completeness
35.9 chromosomes segmented out after crowd annotation, for 50 images having 46 chromosomes
2
% Reduction in Calibration Time
The amount of time saved from manually setting the calibration
2
Number of complaints
reduced % of labor's complaint
improvement of productivity
2
Accuracy
Among all the predicted customer sentiment classification, the ratio of accurate prediction, current value is 76.4%
Increasing to 90%
2
FAR (false acceptance rate)
Negative samples are identified as positive samples / Total number of negative samples.The low FAR, the more smartphone will get correct scenes and objects
Improve accuracy
2
Training efficiency
Amount of necessary data for training might lead to practical obstacles in application.
2
Overall drilling time
The time spent on one drilling job inclusive of the all downtimes
Real Time visibility into operations gives the operations early warnings to take actions immediately.
2
Generation of agents (travel times, speed on link)
Agents generated will build up in the network creating realistic conditionsw of congestion. Speed on links.
Phrase 2
2
split
Proportion of images requiring human inspection. The less the split, the higher the efficiency.
To minimize the human effort in inspection
3
Satisfaction
The number of users who are satisfied with customer service over total number of users
Improve user experience
3
Lead time
time from order to shipment
improvement of productivity
3
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 90%
Increasing to 90%
3
Initial success rate
After initial training, the success rate needs to be acceptable such that the system can be put in the production line.
3
Opeartion of service (number of users for the service)
Optimisation of route and operation time in the day. Validation provided using data collected by Mobility service operators during the operation of service
Phrase 3
3
resource utilization ratio
Achievable resource utilization ratio in the hardware infrastructure ( the higher the utilization ratio, the lower amount the required resource)
To reduce the infrastructure investment and overall solution cost
4
Accuracy
Among all the predicted customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
4
Speed of improvement
Higher convergence speed of the reinforcement algorithm is making the solution more attractive.
5
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
5
Operational efficiency
Cycle time is the primary measure in manufacturing industry.
6
Success rate
Very high success rate is required for the solution to be accepted.
Mandate of the key sensors based on the type of equipment Based on the type of equipment, the makers need to have the basic set on sensors imbibed onto the system. E.g. for a pump – it is important to measure the input flow and output flow rates, vibrations, rotation speed, lube oil temperature and pressure. This will guide the equipment manufactures to provide their customers and their data products to capture the minimum required data and understand the equipment performance
Mandate for the organizations to expose the minimum and key parameters The equipment owners need to enable the basic set of sensors for the equipment health and performance which are required for monitoring the asset from any failures
Standards for Data Formats Each organization has a different way of capturing data and storing them in different formats. Due to which the solutions are not scalable across organizations though the product behind them is same. It takes customised efforts each time.
Guidelines for deciding the sampling frequency based on the type of data We see a need to have a specific set of guidelines to capture data at a minimum required sampling frequency. For example, a vibration sensor should capture data at least at 1 ms or less.
Guidelines for Feature Engineering There must be guidelines as to how the features need to be engineered for AI models. Lack of this would lead to more black box models not explaining how the models behave the way they do.
Guidelines for Standardization of event types and codes There are multiple events which occur for an asset or in a manufacturing plant. Guidelines would help people capture the data in a similar fashion helping the industry to benchmark against one another and at industry level we can understand, which events are the most critical. Guidelines for standardization of Fault and Error Codes for an equipment or process Similar to events, it is also useful to capture fault, failure and error codes in a standard way.
Process Guidelines for event related data (Maintenance and Work Orders) Guidelines would help people capture the data in a similar fashion helping the industry to benchmark against one another and at industry level we can understand, which events are the most critical
Guidelines for Training AI models A defined set of guidelines for AI models would be useful for the data scientists to follow. It will also aid the consumers of AI models to understand how the outcome has been deduced
Guidelines around AI model explainability With so many black box models floating around in the industry, it is difficult for consumers of AI models to understand then and their output. And with engineers and domain experts, coming into the picture, it is very much required to make these models more explainable.
Process Guidelines and methods for model evaluation (retraining) Before deployment and post deployment, it is very critical to have standard methods for models. And also post deployment, we must set guidelines for retaining the model on a periodic basis or based on data volatility. This is increasingly becoming important as AI models are being involved in more strategic and operational decision making.
Guidelines for disaster recovery and autonomous operations With the aid of AI models, the operations of an equipment or manufacturing plant are becoming more and more autonomous and self- sufficient. But the human monitoring is also important as any kind of inaccurate prediction can lead to a disaster and it is must to have some standard to recover from this situation and to assess the conditions to go for autonomous operations.
Editor's comments and enhancements are shown in green. [✓ Reviewed]
The quality of use case submissions will be evaluated for inclusion in the Working Group's Technical Report based on the application area, relevant AI technologies, credible reference sources (see References section), and the following characteristics:
[1] Data Focus & Learning: Use cases for AI system which utilizes Machine Learning, and those that use a fixed a priori knowledge base.
[2] Level of Autonomy: Use cases demonstrating several degrees (dependent, autonomous, human/critic in the loop, etc.) of AI system autonomy.
[3] Verifiability & Transparency: Use cases demonstrating several types and levels of verifiability and transparency, including approaches for explainable AI, accountability, etc.
[4] Impact: Use cases demonstrating the impact of AI systems to society, environment, etc.
[5] Architecture: Use cases demonstrating several architectural paradigms for AI systems (e.g., cloud, distributed AI, crowdsourcing, swarm intelligence, etc.)
[6] Functional aspects, trustworthiness, and societal concerns
[7] AI life cycle components include acquire/process/apply.
These characteristics are identified in red in the use case.
To improve productivity of warehouse operation by detecting and changing controllable factors
Short Description (up to 150 words)
AI-driven operating system that uses big data from work performance information to issue appropriate work instructions has been developed. In PoC, picking operation improvement was conducted in a distribution warehouse. As the result, 8% work reduction was performed.
Complete Description
Attempts are being made to increase the efficiency of work improvements through more widespread application of IT to work systems. However, as each new improvement is added or improvements are made with respect to environmental changes, it requires manual changes to the system, leading to increases in work improvement costs. This case has developed an AI system that uses big data such as work performance information, to understand worksite improvements and environmental changes and issue appropriate work instructions. It has conducted a demonstration test, which confirmed the effectiveness of this system for improving distribution warehouse work. In the future, we will continue to work on expanding the AI system to a wide range areas such as manufacturing and distribution.
Ratio of potential issues which are "of interest" for human evaluation. Ideal target is to reduce the current volume by 80%.
Improve accuracy
1
Precision
Correctly Predicted Anomalous scenarios/ Total Anomalous scenarios predicted
1
Features related to adulterants in radio spectrum
Intensities around NIR range
1
Average vehicle driving speed
Average vehicle driving speed on all the road sections in a given region
Improve the road utilization efficiency
1
Coverage
Ratio of EMR QC requirements done in the solution/all issued EMR QC requirements in China. Ideal target is 100%.
Improve accuracy
1
Accuracy
The number of correctly recognized users’ intent over total number of users. Currently, accuracy reaches 95%.
Improve accuracy of recognizing users’ intent
1
Classifier Accuracy
Without straightening and pre-processing, the average classification accuracy obtained was 68.5%. However, with preprocessing, the classification accuracy improved to 86.7%. These results are very likely to improve with more annotated training data for classification.
1
Closeness to Golden Batch
How close a process is to the best possible batch
Helps in isolation of bad batches from good batches by identifying combination of process variable trajectories that lead to good or bad batch operation.
1
Model Accuracy
Accuracy of the prediction model
The extent to which the setpoints have correctly predicted
1
Ratio of ML discovered failure rate to nominal failure rate
What combination of manufacturing processes/decisions leads to higher failure rates compared to nominal failure rate
Actionable intelligence to improve the manufacturing process of HV circuit breakers
1
Number of labors
reduced % of labors
improvement of productivity
1
Customer Satisfaction
The ratio of customer satisfaction when using this system for requests. The expectation is 100%
Increasing its ratio as high as possible
1
MIoU (Mean Intersection over Union)
The intersection of prediction area and actual area divided by the union of the predicted area and the actual area. Ideal target is 100%.
Improve accuracy
1
Classification Ratio
Real to Pseudo wrong classification
Establishes the quality of identification
1
Ease of use
Simplicity and efficiency during initial learning. Teaching process should be easy.
1
Zone of Influence/ Thermal Correlation Index
Extent of influence of ACUs on data center racks.
Helps in improved control.
1
Invisible Loss Time
Indicates the lost time of the asset in being idle or off or unplanned downtime
Asset Utilization Reports indicate the effectively utilized time there indicating the lost time and their causes
1
Prediction Accuracy
To what extent has the model been able to predict correctly
Provided ability as to % of times the quality complied
1
Algorithm accuracy
Output when compared to the human expert analysis of the same data
See Reference
1
Generation of Activities (land use information and time of travel)
Purpose of activities is assigned based on land use information and time of travel. Census data and national/ local travel surveys will provide validation for the process
Phrase 1
1
accuracy
The accuracy of infraction and incident detection from traffic pictures/videos
To increase the accuracy of traffic monitoring and inspection
2
Split
Proportion of the potential issues which are "more likely to be a valid issue" for our end users.
Average vehicle waiting time at all the intersections in a given region
Improve the road utilization efficiency
2
Resolution
The number of answers solved over total number of questions asked
Improve the resolution of questions from users
2
Annotation Completeness
35.9 chromosomes segmented out after crowd annotation, for 50 images having 46 chromosomes
2
% Reduction in Calibration Time
The amount of time saved from manually setting the calibration
2
Number of complaints
reduced % of labor's complaint
improvement of productivity
2
Accuracy
Among all the predicted customer sentiment classification, the ratio of accurate prediction, current value is 76.4%
Increasing to 90%
2
FAR (false acceptance rate)
Negative samples are identified as positive samples / Total number of negative samples.The low FAR, the more smartphone will get correct scenes and objects
Improve accuracy
2
Training efficiency
Amount of necessary data for training might lead to practical obstacles in application.
2
Overall drilling time
The time spent on one drilling job inclusive of the all downtimes
Real Time visibility into operations gives the operations early warnings to take actions immediately.
2
Generation of agents (travel times, speed on link)
Agents generated will build up in the network creating realistic conditionsw of congestion. Speed on links.
Phrase 2
2
split
Proportion of images requiring human inspection. The less the split, the higher the efficiency.
To minimize the human effort in inspection
3
Satisfaction
The number of users who are satisfied with customer service over total number of users
Improve user experience
3
Lead time
time from order to shipment
improvement of productivity
3
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 90%
Increasing to 90%
3
Initial success rate
After initial training, the success rate needs to be acceptable such that the system can be put in the production line.
3
Opeartion of service (number of users for the service)
Optimisation of route and operation time in the day. Validation provided using data collected by Mobility service operators during the operation of service
Phrase 3
3
resource utilization ratio
Achievable resource utilization ratio in the hardware infrastructure ( the higher the utilization ratio, the lower amount the required resource)
To reduce the infrastructure investment and overall solution cost
4
Accuracy
Among all the predicted customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
4
Speed of improvement
Higher convergence speed of the reinforcement algorithm is making the solution more attractive.
5
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
5
Operational efficiency
Cycle time is the primary measure in manufacturing industry.
6
Success rate
Very high success rate is required for the solution to be accepted.
Editor's comments and enhancements are shown in green. [✓ Reviewed]
The quality of use case submissions will be evaluated for inclusion in the Working Group's Technical Report based on the application area, relevant AI technologies, credible reference sources (see References section), and the following characteristics:
[1] Data Focus & Learning: Use cases for AI system which utilizes Machine Learning, and those that use a fixed a priori knowledge base.
[2] Level of Autonomy: Use cases demonstrating several degrees (dependent, autonomous, human/critic in the loop, etc.) of AI system autonomy.
[3] Verifiability & Transparency: Use cases demonstrating several types and levels of verifiability and transparency, including approaches for explainable AI, accountability, etc.
[4] Impact: Use cases demonstrating the impact of AI systems to society, environment, etc.
[5] Architecture: Use cases demonstrating several architectural paradigms for AI systems (e.g., cloud, distributed AI, crowdsourcing, swarm intelligence, etc.)
[6] Functional aspects, trustworthiness, and societal concerns
[7] AI life cycle components include acquire/process/apply.
These characteristics are identified in red in the use case.
Extracting sentiment and its intensity from customers’ input, and responding with appropriate attitude in order to improve the quality of customers’ inquiry.
To design an efficient solution for customers’ sentiment and intensity detection, especially in the situation of limited training dataset.
Short Description (up to 150 words)
The emotion-sensitive AI customer service of JD.com Int., is supported by AI technology and deep learning method. It is developed for ameliorating accuracy of customer sentiment and intensity. In sentiment classification, it has achieved 74% accuracy and 90% recall score while in intensity detection, it has accomplished 85% accuracy and 85% recall. During the special sale of -618-, it has increased customer satisfaction by 57%.
Complete Description
JD’s customer service representatives need to handle millions of requests on a daily basis. Regular AI customer service systems, 24/7 online, are capable of offering instant assistance, which alleviates the labor resources to a large extent. However, it is quite challenging, if not impossible, for those systems to interpret emotions from customer input and respond as friendly as human. Under this background, based on huge data set of customer comments and rich experience of Natural Language Processing, our system can automatically detect sentiments like happy, angry, anxious, etc. Moreover, this system can also detect the intensity of customer sentiment. Furthermore, we adapt Convolutional Neural Networks, a widely used techniques in visual computing, to interpret the semantic meaning of customer’s expression. It can improve the system’s performance for sentiment classification and intensity detection. Moreover, with the adoption of transfer learning, the system can also be applied into various types of data. To overcome the difficulty of limited training data, we also use data augmentation method such as reverse translation and data noise to increase the variability of training data. Up to now, the system has reached 90% recall and 74% accuracy rate for sentiment classification over 7 categories. The overall recall and accuracy for sentiment intensity are also around 85%?it has increased customer satisfaction by 57%.
The low degree of humanization, and lack of semantic diversity for response. Reducing the number of human customer service.
Performance Indicators (KPIs)
Seq. No.
Name
Description
Reference to mentioned use case objectives
1
Coverage
Ratio of potential issues which are "of interest" for human evaluation. Ideal target is to reduce the current volume by 80%.
Improve accuracy
1
Precision
Correctly Predicted Anomalous scenarios/ Total Anomalous scenarios predicted
1
Features related to adulterants in radio spectrum
Intensities around NIR range
1
Average vehicle driving speed
Average vehicle driving speed on all the road sections in a given region
Improve the road utilization efficiency
1
Coverage
Ratio of EMR QC requirements done in the solution/all issued EMR QC requirements in China. Ideal target is 100%.
Improve accuracy
1
Accuracy
The number of correctly recognized users’ intent over total number of users. Currently, accuracy reaches 95%.
Improve accuracy of recognizing users’ intent
1
Classifier Accuracy
Without straightening and pre-processing, the average classification accuracy obtained was 68.5%. However, with preprocessing, the classification accuracy improved to 86.7%. These results are very likely to improve with more annotated training data for classification.
1
Closeness to Golden Batch
How close a process is to the best possible batch
Helps in isolation of bad batches from good batches by identifying combination of process variable trajectories that lead to good or bad batch operation.
1
Model Accuracy
Accuracy of the prediction model
The extent to which the setpoints have correctly predicted
1
Ratio of ML discovered failure rate to nominal failure rate
What combination of manufacturing processes/decisions leads to higher failure rates compared to nominal failure rate
Actionable intelligence to improve the manufacturing process of HV circuit breakers
1
Number of labors
reduced % of labors
improvement of productivity
1
Customer Satisfaction
The ratio of customer satisfaction when using this system for requests. The expectation is 100%
Increasing its ratio as high as possible
1
MIoU (Mean Intersection over Union)
The intersection of prediction area and actual area divided by the union of the predicted area and the actual area. Ideal target is 100%.
Improve accuracy
1
Classification Ratio
Real to Pseudo wrong classification
Establishes the quality of identification
1
Ease of use
Simplicity and efficiency during initial learning. Teaching process should be easy.
1
Zone of Influence/ Thermal Correlation Index
Extent of influence of ACUs on data center racks.
Helps in improved control.
1
Invisible Loss Time
Indicates the lost time of the asset in being idle or off or unplanned downtime
Asset Utilization Reports indicate the effectively utilized time there indicating the lost time and their causes
1
Prediction Accuracy
To what extent has the model been able to predict correctly
Provided ability as to % of times the quality complied
1
Algorithm accuracy
Output when compared to the human expert analysis of the same data
See Reference
1
Generation of Activities (land use information and time of travel)
Purpose of activities is assigned based on land use information and time of travel. Census data and national/ local travel surveys will provide validation for the process
Phrase 1
1
accuracy
The accuracy of infraction and incident detection from traffic pictures/videos
To increase the accuracy of traffic monitoring and inspection
2
Split
Proportion of the potential issues which are "more likely to be a valid issue" for our end users.
Average vehicle waiting time at all the intersections in a given region
Improve the road utilization efficiency
2
Resolution
The number of answers solved over total number of questions asked
Improve the resolution of questions from users
2
Annotation Completeness
35.9 chromosomes segmented out after crowd annotation, for 50 images having 46 chromosomes
2
% Reduction in Calibration Time
The amount of time saved from manually setting the calibration
2
Number of complaints
reduced % of labor's complaint
improvement of productivity
2
Accuracy
Among all the predicted customer sentiment classification, the ratio of accurate prediction, current value is 76.4%
Increasing to 90%
2
FAR (false acceptance rate)
Negative samples are identified as positive samples / Total number of negative samples.The low FAR, the more smartphone will get correct scenes and objects
Improve accuracy
2
Training efficiency
Amount of necessary data for training might lead to practical obstacles in application.
2
Overall drilling time
The time spent on one drilling job inclusive of the all downtimes
Real Time visibility into operations gives the operations early warnings to take actions immediately.
2
Generation of agents (travel times, speed on link)
Agents generated will build up in the network creating realistic conditionsw of congestion. Speed on links.
Phrase 2
2
split
Proportion of images requiring human inspection. The less the split, the higher the efficiency.
To minimize the human effort in inspection
3
Satisfaction
The number of users who are satisfied with customer service over total number of users
Improve user experience
3
Lead time
time from order to shipment
improvement of productivity
3
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 90%
Increasing to 90%
3
Initial success rate
After initial training, the success rate needs to be acceptable such that the system can be put in the production line.
3
Opeartion of service (number of users for the service)
Optimisation of route and operation time in the day. Validation provided using data collected by Mobility service operators during the operation of service
Phrase 3
3
resource utilization ratio
Achievable resource utilization ratio in the hardware infrastructure ( the higher the utilization ratio, the lower amount the required resource)
To reduce the infrastructure investment and overall solution cost
4
Accuracy
Among all the predicted customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
4
Speed of improvement
Higher convergence speed of the reinforcement algorithm is making the solution more attractive.
5
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
5
Operational efficiency
Cycle time is the primary measure in manufacturing industry.
6
Success rate
Very high success rate is required for the solution to be accepted.
Deep learning: a class of machine learning algorithms use a cascade of multiple layers of nonlinear processing units for feature extraction and transformation. Transfer learning: we adopt multi-task learning method in this system. Jointly training different annotated data in same domain, this method improves the model performance for classification problems. Data augmentation: we apply reverse translation to firstly translation Chinese into English and then translate it backward. We also use data noise to improve the data diversity.
Standardization Opportunities Requirements
The system can be promoted to as many customer cervices companies as possible once provide with enough training data for the specific Application scenario
Challenges & Issues
Challenge: the system’s performance should be as good as the human customer server. Issues: 1) limited training data; 2) sentiment classification among seven categories.
For sentiment classification: conversation data from after-sales customer services. It’s annotated by professional annotators into 7 categories of sentiments. For sentiment intensity: Only including sentiment data with “anger” and “anxious”; it’s annotated into 3 degrees of intensity: “low, medium, high”.
Editor's comments and enhancements are shown in green. [✓ Reviewed]
The quality of use case submissions will be evaluated for inclusion in the Working Group's Technical Report based on the application area, relevant AI technologies, credible reference sources (see References section), and the following characteristics:
[1] Data Focus & Learning: Use cases for AI system which utilizes Machine Learning, and those that use a fixed a priori knowledge base.
[2] Level of Autonomy: Use cases demonstrating several degrees (dependent, autonomous, human/critic in the loop, etc.) of AI system autonomy.
[3] Verifiability & Transparency: Use cases demonstrating several types and levels of verifiability and transparency, including approaches for explainable AI, accountability, etc.
[4] Impact: Use cases demonstrating the impact of AI systems to society, environment, etc.
[5] Architecture: Use cases demonstrating several architectural paradigms for AI systems (e.g., cloud, distributed AI, crowdsourcing, swarm intelligence, etc.)
[6] Functional aspects, trustworthiness, and societal concerns
[7] AI life cycle components include acquire/process/apply.
These characteristics are identified in red in the use case.
No.
32
ID:
Use Case Name:
AI solution to help mobile phone to have better picture effect
To find an efficient solution to Increase camera image quality on smartphone without Increasing too much operation and power burden for mobile phone.
Short Description (up to 150 words)
An AI solution was developed that could increase smartphone camera image quality. Using deep learning, smartphone can Identify more scenarios and objects than before. Based on the identified scenarios and objects, smartphone can better understand the image and improve image effect.
Complete Description
At present, there are 1.4 billion smart phone shipments in the world every year. Photography is one of the most important functions of smart phones. The industry has been trying to improve the picture quality of mobile phone photography. It hopes to reach even the quality of the professional SLR camera. The traditional image processing algorithm is currently facing the ceiling, many scenes traditional algorithms can ot be used, just because the effect is very poor.
Deep learning algorithm provides a turning point for solving the above problems. By using the AI solution, smartphones can better "understand" the pictures they take. Based on the deep learning algorithm, the smart phone can analyze the shooting scene in real time and intelligently identify various scenes in the shooting process, such as blue sky, flowers, green plants, night view, snow scene, etc. And the smart phone can also intelligently detect the shooting objects in the scene. Base on scene recognition and object detection,the smartphone can automatically adjust and set parameters for different pictures, so as to get better photo effects. Now the mobile phone can recognize 100 kinds of scenes and can reach hundreds in the future. By using the depth learning algorithm, the mobile phone can now detect the 20 types of subjects, and the future can be detected by hundreds of subjects. Object detection can be used for SmartZoom (auto focus on targets), and portrait segmentation can be used for background blur or light efficiency.
Ratio of potential issues which are "of interest" for human evaluation. Ideal target is to reduce the current volume by 80%.
Improve accuracy
1
Precision
Correctly Predicted Anomalous scenarios/ Total Anomalous scenarios predicted
1
Features related to adulterants in radio spectrum
Intensities around NIR range
1
Average vehicle driving speed
Average vehicle driving speed on all the road sections in a given region
Improve the road utilization efficiency
1
Coverage
Ratio of EMR QC requirements done in the solution/all issued EMR QC requirements in China. Ideal target is 100%.
Improve accuracy
1
Accuracy
The number of correctly recognized users’ intent over total number of users. Currently, accuracy reaches 95%.
Improve accuracy of recognizing users’ intent
1
Classifier Accuracy
Without straightening and pre-processing, the average classification accuracy obtained was 68.5%. However, with preprocessing, the classification accuracy improved to 86.7%. These results are very likely to improve with more annotated training data for classification.
1
Closeness to Golden Batch
How close a process is to the best possible batch
Helps in isolation of bad batches from good batches by identifying combination of process variable trajectories that lead to good or bad batch operation.
1
Model Accuracy
Accuracy of the prediction model
The extent to which the setpoints have correctly predicted
1
Ratio of ML discovered failure rate to nominal failure rate
What combination of manufacturing processes/decisions leads to higher failure rates compared to nominal failure rate
Actionable intelligence to improve the manufacturing process of HV circuit breakers
1
Number of labors
reduced % of labors
improvement of productivity
1
Customer Satisfaction
The ratio of customer satisfaction when using this system for requests. The expectation is 100%
Increasing its ratio as high as possible
1
MIoU (Mean Intersection over Union)
The intersection of prediction area and actual area divided by the union of the predicted area and the actual area. Ideal target is 100%.
Improve accuracy
1
Classification Ratio
Real to Pseudo wrong classification
Establishes the quality of identification
1
Ease of use
Simplicity and efficiency during initial learning. Teaching process should be easy.
1
Zone of Influence/ Thermal Correlation Index
Extent of influence of ACUs on data center racks.
Helps in improved control.
1
Invisible Loss Time
Indicates the lost time of the asset in being idle or off or unplanned downtime
Asset Utilization Reports indicate the effectively utilized time there indicating the lost time and their causes
1
Prediction Accuracy
To what extent has the model been able to predict correctly
Provided ability as to % of times the quality complied
1
Algorithm accuracy
Output when compared to the human expert analysis of the same data
See Reference
1
Generation of Activities (land use information and time of travel)
Purpose of activities is assigned based on land use information and time of travel. Census data and national/ local travel surveys will provide validation for the process
Phrase 1
1
accuracy
The accuracy of infraction and incident detection from traffic pictures/videos
To increase the accuracy of traffic monitoring and inspection
2
Split
Proportion of the potential issues which are "more likely to be a valid issue" for our end users.
Average vehicle waiting time at all the intersections in a given region
Improve the road utilization efficiency
2
Resolution
The number of answers solved over total number of questions asked
Improve the resolution of questions from users
2
Annotation Completeness
35.9 chromosomes segmented out after crowd annotation, for 50 images having 46 chromosomes
2
% Reduction in Calibration Time
The amount of time saved from manually setting the calibration
2
Number of complaints
reduced % of labor's complaint
improvement of productivity
2
Accuracy
Among all the predicted customer sentiment classification, the ratio of accurate prediction, current value is 76.4%
Increasing to 90%
2
FAR (false acceptance rate)
Negative samples are identified as positive samples / Total number of negative samples.The low FAR, the more smartphone will get correct scenes and objects
Improve accuracy
2
Training efficiency
Amount of necessary data for training might lead to practical obstacles in application.
2
Overall drilling time
The time spent on one drilling job inclusive of the all downtimes
Real Time visibility into operations gives the operations early warnings to take actions immediately.
2
Generation of agents (travel times, speed on link)
Agents generated will build up in the network creating realistic conditionsw of congestion. Speed on links.
Phrase 2
2
split
Proportion of images requiring human inspection. The less the split, the higher the efficiency.
To minimize the human effort in inspection
3
Satisfaction
The number of users who are satisfied with customer service over total number of users
Improve user experience
3
Lead time
time from order to shipment
improvement of productivity
3
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 90%
Increasing to 90%
3
Initial success rate
After initial training, the success rate needs to be acceptable such that the system can be put in the production line.
3
Opeartion of service (number of users for the service)
Optimisation of route and operation time in the day. Validation provided using data collected by Mobility service operators during the operation of service
Phrase 3
3
resource utilization ratio
Achievable resource utilization ratio in the hardware infrastructure ( the higher the utilization ratio, the lower amount the required resource)
To reduce the infrastructure investment and overall solution cost
4
Accuracy
Among all the predicted customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
4
Speed of improvement
Higher convergence speed of the reinforcement algorithm is making the solution more attractive.
5
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
5
Operational efficiency
Cycle time is the primary measure in manufacturing industry.
6
Success rate
Very high success rate is required for the solution to be accepted.
Editor's comments and enhancements are shown in green. [✓ Reviewed]
The quality of use case submissions will be evaluated for inclusion in the Working Group's Technical Report based on the application area, relevant AI technologies, credible reference sources (see References section), and the following characteristics:
[1] Data Focus & Learning: Use cases for AI system which utilizes Machine Learning, and those that use a fixed a priori knowledge base.
[2] Level of Autonomy: Use cases demonstrating several degrees (dependent, autonomous, human/critic in the loop, etc.) of AI system autonomy.
[3] Verifiability & Transparency: Use cases demonstrating several types and levels of verifiability and transparency, including approaches for explainable AI, accountability, etc.
[4] Impact: Use cases demonstrating the impact of AI systems to society, environment, etc.
[5] Architecture: Use cases demonstrating several architectural paradigms for AI systems (e.g., cloud, distributed AI, crowdsourcing, swarm intelligence, etc.)
[6] Functional aspects, trustworthiness, and societal concerns
[7] AI life cycle components include acquire/process/apply.
These characteristics are identified in red in the use case.
No.
33
ID:
Use Case Name:
Automated defect classification on product surfaces
Image analytics using a combination of feature extraction and classification of defects on shining surfaces in sanitary industries.
Short Description (up to 150 words)
A vision system that inspects and identifies the defects on water taps in sanitary industries. The system uses a combination of features for an automatic defect classification on product surfaces. All defects (15 types are identified) are classified into two major categories, real-defects and pseudo-defects. The pseudo-defects cause no quality problem; while the real-defects are critical as they might malfunction the final products. The AI system uses Support Vector Machine (SVM) classifier along with the combined features to identify the defect types. With the vision system in place, the quality control process is fully automated without any human intervention.
Complete Description
The proposed vision system has two parts: the hardware part and the software part. The hardware captures the images of product surfaces under a constant illuminating condition. The software is developed to perform image processing tasks and identify defects on product surfaces.
The steps of proposed system include image acquisition, preprocessing, segmentation, feature extraction, classification and post-processing. The system presents two software components: Feature Extraction and Classifier Design. These two modules are implemented independently which can be developed in offline platform and can be integrated into vision system and work online.
As a first step, the feature extraction is critical and guides the extent to which a classifier can distinguish the defects from one class to another. A combination of features is used like geometry (shape, texture), and statistical features of the segmented images. In the second step, a support vector machine classification model is trained to identify the defect types. The classification results obtained by combining Gabor features, Statistical features, and grayscale features showed comparable performances with human evaluations.
Overall, the vision system is modularized with capabilities to self-learn and future extensions.
Incorrect AI System use (AI system affecting quality control); New Security Threats.
Performance Indicators (KPIs)
Seq. No.
Name
Description
Reference to mentioned use case objectives
1
Coverage
Ratio of potential issues which are "of interest" for human evaluation. Ideal target is to reduce the current volume by 80%.
Improve accuracy
1
Precision
Correctly Predicted Anomalous scenarios/ Total Anomalous scenarios predicted
1
Features related to adulterants in radio spectrum
Intensities around NIR range
1
Average vehicle driving speed
Average vehicle driving speed on all the road sections in a given region
Improve the road utilization efficiency
1
Coverage
Ratio of EMR QC requirements done in the solution/all issued EMR QC requirements in China. Ideal target is 100%.
Improve accuracy
1
Accuracy
The number of correctly recognized users’ intent over total number of users. Currently, accuracy reaches 95%.
Improve accuracy of recognizing users’ intent
1
Classifier Accuracy
Without straightening and pre-processing, the average classification accuracy obtained was 68.5%. However, with preprocessing, the classification accuracy improved to 86.7%. These results are very likely to improve with more annotated training data for classification.
1
Closeness to Golden Batch
How close a process is to the best possible batch
Helps in isolation of bad batches from good batches by identifying combination of process variable trajectories that lead to good or bad batch operation.
1
Model Accuracy
Accuracy of the prediction model
The extent to which the setpoints have correctly predicted
1
Ratio of ML discovered failure rate to nominal failure rate
What combination of manufacturing processes/decisions leads to higher failure rates compared to nominal failure rate
Actionable intelligence to improve the manufacturing process of HV circuit breakers
1
Number of labors
reduced % of labors
improvement of productivity
1
Customer Satisfaction
The ratio of customer satisfaction when using this system for requests. The expectation is 100%
Increasing its ratio as high as possible
1
MIoU (Mean Intersection over Union)
The intersection of prediction area and actual area divided by the union of the predicted area and the actual area. Ideal target is 100%.
Improve accuracy
1
Classification Ratio
Real to Pseudo wrong classification
Establishes the quality of identification
1
Ease of use
Simplicity and efficiency during initial learning. Teaching process should be easy.
1
Zone of Influence/ Thermal Correlation Index
Extent of influence of ACUs on data center racks.
Helps in improved control.
1
Invisible Loss Time
Indicates the lost time of the asset in being idle or off or unplanned downtime
Asset Utilization Reports indicate the effectively utilized time there indicating the lost time and their causes
1
Prediction Accuracy
To what extent has the model been able to predict correctly
Provided ability as to % of times the quality complied
1
Algorithm accuracy
Output when compared to the human expert analysis of the same data
See Reference
1
Generation of Activities (land use information and time of travel)
Purpose of activities is assigned based on land use information and time of travel. Census data and national/ local travel surveys will provide validation for the process
Phrase 1
1
accuracy
The accuracy of infraction and incident detection from traffic pictures/videos
To increase the accuracy of traffic monitoring and inspection
2
Split
Proportion of the potential issues which are "more likely to be a valid issue" for our end users.
Average vehicle waiting time at all the intersections in a given region
Improve the road utilization efficiency
2
Resolution
The number of answers solved over total number of questions asked
Improve the resolution of questions from users
2
Annotation Completeness
35.9 chromosomes segmented out after crowd annotation, for 50 images having 46 chromosomes
2
% Reduction in Calibration Time
The amount of time saved from manually setting the calibration
2
Number of complaints
reduced % of labor's complaint
improvement of productivity
2
Accuracy
Among all the predicted customer sentiment classification, the ratio of accurate prediction, current value is 76.4%
Increasing to 90%
2
FAR (false acceptance rate)
Negative samples are identified as positive samples / Total number of negative samples.The low FAR, the more smartphone will get correct scenes and objects
Improve accuracy
2
Training efficiency
Amount of necessary data for training might lead to practical obstacles in application.
2
Overall drilling time
The time spent on one drilling job inclusive of the all downtimes
Real Time visibility into operations gives the operations early warnings to take actions immediately.
2
Generation of agents (travel times, speed on link)
Agents generated will build up in the network creating realistic conditionsw of congestion. Speed on links.
Phrase 2
2
split
Proportion of images requiring human inspection. The less the split, the higher the efficiency.
To minimize the human effort in inspection
3
Satisfaction
The number of users who are satisfied with customer service over total number of users
Improve user experience
3
Lead time
time from order to shipment
improvement of productivity
3
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 90%
Increasing to 90%
3
Initial success rate
After initial training, the success rate needs to be acceptable such that the system can be put in the production line.
3
Opeartion of service (number of users for the service)
Optimisation of route and operation time in the day. Validation provided using data collected by Mobility service operators during the operation of service
Phrase 3
3
resource utilization ratio
Achievable resource utilization ratio in the hardware infrastructure ( the higher the utilization ratio, the lower amount the required resource)
To reduce the infrastructure investment and overall solution cost
4
Accuracy
Among all the predicted customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
4
Speed of improvement
Higher convergence speed of the reinforcement algorithm is making the solution more attractive.
5
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
5
Operational efficiency
Cycle time is the primary measure in manufacturing industry.
6
Success rate
Very high success rate is required for the solution to be accepted.
Quality acceptance criterion from AI systems: What is the acceptable standard for AI output related to quality? How that can be independently validated?
Standards for dealing with AI failures: How/Can standards facilitate dealing with AI failures, w.r.t., quality, productivity criteria?
Challenges & Issues
Real time implementation, accurately identify the nature of defects
Editor's comments and enhancements are shown in green. [✓ Reviewed]
The quality of use case submissions will be evaluated for inclusion in the Working Group's Technical Report based on the application area, relevant AI technologies, credible reference sources (see References section), and the following characteristics:
[1] Data Focus & Learning: Use cases for AI system which utilizes Machine Learning, and those that use a fixed a priori knowledge base.
[2] Level of Autonomy: Use cases demonstrating several degrees (dependent, autonomous, human/critic in the loop, etc.) of AI system autonomy.
[3] Verifiability & Transparency: Use cases demonstrating several types and levels of verifiability and transparency, including approaches for explainable AI, accountability, etc.
[4] Impact: Use cases demonstrating the impact of AI systems to society, environment, etc.
[5] Architecture: Use cases demonstrating several architectural paradigms for AI systems (e.g., cloud, distributed AI, crowdsourcing, swarm intelligence, etc.)
[6] Functional aspects, trustworthiness, and societal concerns
[7] AI life cycle components include acquire/process/apply.
These characteristics are identified in red in the use case.
Simple programing/instruction and flexibility in usage
Automation of tasks lacking analytic description
Reliability and efficiency
Short Description (up to 150 words)
Assembly process often includes steps where two parts need to be matched and connected to each other through force exertion. In an ideal case, perfectly formed parts can be matched and be assembled together with predefined amount of force. Due to imperfection of production steps, surface imperfection and other factors such as flexibility of parts, this procedure can become complex and unpredictable. In such cases, human operator can be instructed with simple terms and demonstrations and perform the task easily, while a robotic system will need very detailed and extensive program instructions to be able to perform the task including required adaptation to the physical world. The need for such a complex program instruction will make use of automation cumbersome or uneconomical. Control algorithm that are based on machine learning, especially those including reinforcement learning can become alternative solutions increasing and extending the level of automation in manufacturing.
Complete Description
The case described here is a common step in assembly processes in manufacturing industry and includes matching and properly connecting two parts when one needs to be inserted into another. Successful and efficient insertion usually needs action by feeling. It is difficult to describe in terms of mathematical algorithms and therefore is difficult to program. Complexities in programming, or high degree of operational failure make usage of robots, or automation unattractive. Use of machine learning and artificial intelligence is one of promising methods to overcome such difficulties.
As will be described below, there are several different phases in the process, where different methodologies can and should be used. To make the methodology usable in a practical case, it should be utilizable by operators without deep technical knowledge with an effort that can be accepted on a production line. Ultimately, such methods must remove the need for programing completely.
The assumption here is that the parts to be assembled are properly localized, such that they can be manipulated by a robot in the desired way. The problem concerns the following steps:
Identification and picking the first part (A).
Moving A to the vicinity of the second part (B).
Alignment of the two parts.
Exertion of force with simultaneous movement for smooth insertion.
Termination of the task when complete insertion is complete.
The above task, with all possible challenges, can easily be performed by a human operator. An operator in majority of cases needs very limited amount of information. Using prior knowledge and experiences and the sensory system the task can be completed and all possible exceptions can be handled. With time, a human operator becomes constantly more efficient and performs the task faster and more reliably.
The topics to be handled in this use case are how a machine can be instructed, trained, perform and improve to a high level of reliability and efficiency. The process can be divided into following steps:
Localization of parts: Image processing, object identification, classification and localization.
Alignment of parts: Control and optimization with (mainly) vision inputs.
Insertion through exertion of forces: Control and optimization with (at least) vision and force sensor feedback
Sensing the termination of the process: Pattern recognition in time series.
Continuous improvement: Reinforcement learning.
Vision and force sensors are most commonly used sensors in such processes. The objects and environment need to be observed at moderate as well as in very close distances. Force sensors are needed but have the weakness of not being active before a complete contact. Therefore, use of other sensors could be helpful.
The method is used for assembly tasks with the target of reducing the programming effort and increasing flexibility. For that to be achieved, the effort necessary to teach, train and use the system should be minimum and the reliability should come high at short time. This implicitly means that the system should become useful with limited amount of data and at limited amount of time. After an initial relatively stable state is reached, reinforcement can be used to improve the efficiency of the system.
The solution will become more attractive if transfer learning is utilized to further reduce the initial training time.
For benchmarking purpose a specific set of objects to be assembled together should be defined and performance of the methods can be measured by necessary training time, need for computing power and memory as well as time for completion of the task. The objects in the tests can be geometrically relatively simple. Special features such as rough surfaces, tight fitting or flexibility of the objects can be considered for different classes of problems.
Ratio of potential issues which are "of interest" for human evaluation. Ideal target is to reduce the current volume by 80%.
Improve accuracy
1
Precision
Correctly Predicted Anomalous scenarios/ Total Anomalous scenarios predicted
1
Features related to adulterants in radio spectrum
Intensities around NIR range
1
Average vehicle driving speed
Average vehicle driving speed on all the road sections in a given region
Improve the road utilization efficiency
1
Coverage
Ratio of EMR QC requirements done in the solution/all issued EMR QC requirements in China. Ideal target is 100%.
Improve accuracy
1
Accuracy
The number of correctly recognized users’ intent over total number of users. Currently, accuracy reaches 95%.
Improve accuracy of recognizing users’ intent
1
Classifier Accuracy
Without straightening and pre-processing, the average classification accuracy obtained was 68.5%. However, with preprocessing, the classification accuracy improved to 86.7%. These results are very likely to improve with more annotated training data for classification.
1
Closeness to Golden Batch
How close a process is to the best possible batch
Helps in isolation of bad batches from good batches by identifying combination of process variable trajectories that lead to good or bad batch operation.
1
Model Accuracy
Accuracy of the prediction model
The extent to which the setpoints have correctly predicted
1
Ratio of ML discovered failure rate to nominal failure rate
What combination of manufacturing processes/decisions leads to higher failure rates compared to nominal failure rate
Actionable intelligence to improve the manufacturing process of HV circuit breakers
1
Number of labors
reduced % of labors
improvement of productivity
1
Customer Satisfaction
The ratio of customer satisfaction when using this system for requests. The expectation is 100%
Increasing its ratio as high as possible
1
MIoU (Mean Intersection over Union)
The intersection of prediction area and actual area divided by the union of the predicted area and the actual area. Ideal target is 100%.
Improve accuracy
1
Classification Ratio
Real to Pseudo wrong classification
Establishes the quality of identification
1
Ease of use
Simplicity and efficiency during initial learning. Teaching process should be easy.
1
Zone of Influence/ Thermal Correlation Index
Extent of influence of ACUs on data center racks.
Helps in improved control.
1
Invisible Loss Time
Indicates the lost time of the asset in being idle or off or unplanned downtime
Asset Utilization Reports indicate the effectively utilized time there indicating the lost time and their causes
1
Prediction Accuracy
To what extent has the model been able to predict correctly
Provided ability as to % of times the quality complied
1
Algorithm accuracy
Output when compared to the human expert analysis of the same data
See Reference
1
Generation of Activities (land use information and time of travel)
Purpose of activities is assigned based on land use information and time of travel. Census data and national/ local travel surveys will provide validation for the process
Phrase 1
1
accuracy
The accuracy of infraction and incident detection from traffic pictures/videos
To increase the accuracy of traffic monitoring and inspection
2
Split
Proportion of the potential issues which are "more likely to be a valid issue" for our end users.
Average vehicle waiting time at all the intersections in a given region
Improve the road utilization efficiency
2
Resolution
The number of answers solved over total number of questions asked
Improve the resolution of questions from users
2
Annotation Completeness
35.9 chromosomes segmented out after crowd annotation, for 50 images having 46 chromosomes
2
% Reduction in Calibration Time
The amount of time saved from manually setting the calibration
2
Number of complaints
reduced % of labor's complaint
improvement of productivity
2
Accuracy
Among all the predicted customer sentiment classification, the ratio of accurate prediction, current value is 76.4%
Increasing to 90%
2
FAR (false acceptance rate)
Negative samples are identified as positive samples / Total number of negative samples.The low FAR, the more smartphone will get correct scenes and objects
Improve accuracy
2
Training efficiency
Amount of necessary data for training might lead to practical obstacles in application.
2
Overall drilling time
The time spent on one drilling job inclusive of the all downtimes
Real Time visibility into operations gives the operations early warnings to take actions immediately.
2
Generation of agents (travel times, speed on link)
Agents generated will build up in the network creating realistic conditionsw of congestion. Speed on links.
Phrase 2
2
split
Proportion of images requiring human inspection. The less the split, the higher the efficiency.
To minimize the human effort in inspection
3
Satisfaction
The number of users who are satisfied with customer service over total number of users
Improve user experience
3
Lead time
time from order to shipment
improvement of productivity
3
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 90%
Increasing to 90%
3
Initial success rate
After initial training, the success rate needs to be acceptable such that the system can be put in the production line.
3
Opeartion of service (number of users for the service)
Optimisation of route and operation time in the day. Validation provided using data collected by Mobility service operators during the operation of service
Phrase 3
3
resource utilization ratio
Achievable resource utilization ratio in the hardware infrastructure ( the higher the utilization ratio, the lower amount the required resource)
To reduce the infrastructure investment and overall solution cost
4
Accuracy
Among all the predicted customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
4
Speed of improvement
Higher convergence speed of the reinforcement algorithm is making the solution more attractive.
5
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
5
Operational efficiency
Cycle time is the primary measure in manufacturing industry.
6
Success rate
Very high success rate is required for the solution to be accepted.
Standardization of fail-safe options w.r.t. safety and quality;
Standardization towards “Human-Co-working”
Minimum acceptable standards for commercialization;
Standard data set to independently validate the claims
Challenges & Issues
Complex and unpredictable assembly process due to imperfection of production steps, surface imperfection and other factors such as flexibility of parts.
Editor's comments and enhancements are shown in green. [✓ Reviewed]
The quality of use case submissions will be evaluated for inclusion in the Working Group's Technical Report based on the application area, relevant AI technologies, credible reference sources (see References section), and the following characteristics:
[1] Data Focus & Learning: Use cases for AI system which utilizes Machine Learning, and those that use a fixed a priori knowledge base.
[2] Level of Autonomy: Use cases demonstrating several degrees (dependent, autonomous, human/critic in the loop, etc.) of AI system autonomy.
[3] Verifiability & Transparency: Use cases demonstrating several types and levels of verifiability and transparency, including approaches for explainable AI, accountability, etc.
[4] Impact: Use cases demonstrating the impact of AI systems to society, environment, etc.
[5] Architecture: Use cases demonstrating several architectural paradigms for AI systems (e.g., cloud, distributed AI, crowdsourcing, swarm intelligence, etc.)
[6] Functional aspects, trustworthiness, and societal concerns
[7] AI life cycle components include acquire/process/apply.
These characteristics are identified in red in the use case.
No.
35
ID:
Use Case Name:
Causality-based Thermal Prediction for Data Center
Data centers tend to be overcooled to prevent computing machines from failing due to heat. A reliable fine-grained control that could regulate air control unit (ACU) supply air temperature or flow is needed to avoid overcooling. Methods that are based on correlation-based techniques do not generalize well. Hence, we seek to uncover the causal relationship between ACUs supplying cool air and temperature at the cabinets to prioritize which ACUs should be regulated to control a hot-spot near a cabinet
Complete Description
First, we perform experiments in 6SigmaRoom for the layout of the data center being studied. We collect time-series data for supply air temperature and flow per ACU, and for inlet temperature at the cabinets. Next, we test the recorded time series for checking if Granger-causality (G-causality) can be established between the supply air temperature from an ACU to a cabinet. G-causality establishes the unidirectional temporal precedence for data center control actions from ACUs that leads to changes in specific cabinet temperatures. A variable X is said to Granger-Cause Y if, including data about past terms from X, leads to a better prediction of the future value of Y (i.e., Yt+1) than predicting Yt+1 based solely on past terms from Y.
We show by way of simulation that the ACU flows that Granger-Cause reduction in temperature at a cabinet provide a larger share of influence (based on Zone of Influence/Thermal Correlation Index from the simulation) on the cabinet. This could allow an operator to come up with a better control strategy to control hotspots in a data center by regulating ACU supply air temperature/flows.
Ratio of potential issues which are "of interest" for human evaluation. Ideal target is to reduce the current volume by 80%.
Improve accuracy
1
Precision
Correctly Predicted Anomalous scenarios/ Total Anomalous scenarios predicted
1
Features related to adulterants in radio spectrum
Intensities around NIR range
1
Average vehicle driving speed
Average vehicle driving speed on all the road sections in a given region
Improve the road utilization efficiency
1
Coverage
Ratio of EMR QC requirements done in the solution/all issued EMR QC requirements in China. Ideal target is 100%.
Improve accuracy
1
Accuracy
The number of correctly recognized users’ intent over total number of users. Currently, accuracy reaches 95%.
Improve accuracy of recognizing users’ intent
1
Classifier Accuracy
Without straightening and pre-processing, the average classification accuracy obtained was 68.5%. However, with preprocessing, the classification accuracy improved to 86.7%. These results are very likely to improve with more annotated training data for classification.
1
Closeness to Golden Batch
How close a process is to the best possible batch
Helps in isolation of bad batches from good batches by identifying combination of process variable trajectories that lead to good or bad batch operation.
1
Model Accuracy
Accuracy of the prediction model
The extent to which the setpoints have correctly predicted
1
Ratio of ML discovered failure rate to nominal failure rate
What combination of manufacturing processes/decisions leads to higher failure rates compared to nominal failure rate
Actionable intelligence to improve the manufacturing process of HV circuit breakers
1
Number of labors
reduced % of labors
improvement of productivity
1
Customer Satisfaction
The ratio of customer satisfaction when using this system for requests. The expectation is 100%
Increasing its ratio as high as possible
1
MIoU (Mean Intersection over Union)
The intersection of prediction area and actual area divided by the union of the predicted area and the actual area. Ideal target is 100%.
Improve accuracy
1
Classification Ratio
Real to Pseudo wrong classification
Establishes the quality of identification
1
Ease of use
Simplicity and efficiency during initial learning. Teaching process should be easy.
1
Zone of Influence/ Thermal Correlation Index
Extent of influence of ACUs on data center racks.
Helps in improved control.
1
Invisible Loss Time
Indicates the lost time of the asset in being idle or off or unplanned downtime
Asset Utilization Reports indicate the effectively utilized time there indicating the lost time and their causes
1
Prediction Accuracy
To what extent has the model been able to predict correctly
Provided ability as to % of times the quality complied
1
Algorithm accuracy
Output when compared to the human expert analysis of the same data
See Reference
1
Generation of Activities (land use information and time of travel)
Purpose of activities is assigned based on land use information and time of travel. Census data and national/ local travel surveys will provide validation for the process
Phrase 1
1
accuracy
The accuracy of infraction and incident detection from traffic pictures/videos
To increase the accuracy of traffic monitoring and inspection
2
Split
Proportion of the potential issues which are "more likely to be a valid issue" for our end users.
Average vehicle waiting time at all the intersections in a given region
Improve the road utilization efficiency
2
Resolution
The number of answers solved over total number of questions asked
Improve the resolution of questions from users
2
Annotation Completeness
35.9 chromosomes segmented out after crowd annotation, for 50 images having 46 chromosomes
2
% Reduction in Calibration Time
The amount of time saved from manually setting the calibration
2
Number of complaints
reduced % of labor's complaint
improvement of productivity
2
Accuracy
Among all the predicted customer sentiment classification, the ratio of accurate prediction, current value is 76.4%
Increasing to 90%
2
FAR (false acceptance rate)
Negative samples are identified as positive samples / Total number of negative samples.The low FAR, the more smartphone will get correct scenes and objects
Improve accuracy
2
Training efficiency
Amount of necessary data for training might lead to practical obstacles in application.
2
Overall drilling time
The time spent on one drilling job inclusive of the all downtimes
Real Time visibility into operations gives the operations early warnings to take actions immediately.
2
Generation of agents (travel times, speed on link)
Agents generated will build up in the network creating realistic conditionsw of congestion. Speed on links.
Phrase 2
2
split
Proportion of images requiring human inspection. The less the split, the higher the efficiency.
To minimize the human effort in inspection
3
Satisfaction
The number of users who are satisfied with customer service over total number of users
Improve user experience
3
Lead time
time from order to shipment
improvement of productivity
3
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 90%
Increasing to 90%
3
Initial success rate
After initial training, the success rate needs to be acceptable such that the system can be put in the production line.
3
Opeartion of service (number of users for the service)
Optimisation of route and operation time in the day. Validation provided using data collected by Mobility service operators during the operation of service
Phrase 3
3
resource utilization ratio
Achievable resource utilization ratio in the hardware infrastructure ( the higher the utilization ratio, the lower amount the required resource)
To reduce the infrastructure investment and overall solution cost
4
Accuracy
Among all the predicted customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
4
Speed of improvement
Higher convergence speed of the reinforcement algorithm is making the solution more attractive.
5
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
5
Operational efficiency
Cycle time is the primary measure in manufacturing industry.
6
Success rate
Very high success rate is required for the solution to be accepted.
Editor's comments and enhancements are shown in green. [✓ Reviewed]
The quality of use case submissions will be evaluated for inclusion in the Working Group's Technical Report based on the application area, relevant AI technologies, credible reference sources (see References section), and the following characteristics:
[1] Data Focus & Learning: Use cases for AI system which utilizes Machine Learning, and those that use a fixed a priori knowledge base.
[2] Level of Autonomy: Use cases demonstrating several degrees (dependent, autonomous, human/critic in the loop, etc.) of AI system autonomy.
[3] Verifiability & Transparency: Use cases demonstrating several types and levels of verifiability and transparency, including approaches for explainable AI, accountability, etc.
[4] Impact: Use cases demonstrating the impact of AI systems to society, environment, etc.
[5] Architecture: Use cases demonstrating several architectural paradigms for AI systems (e.g., cloud, distributed AI, crowdsourcing, swarm intelligence, etc.)
[6] Functional aspects, trustworthiness, and societal concerns
[7] AI life cycle components include acquire/process/apply.
These characteristics are identified in red in the use case.
Automatic generation of Daily Performance Report, reduction in overall drilling time, cut down Invisible Loss Time and improve rig asset management
Short Description (up to 150 words)
It is important for a drilling contractor to have real time monitoring of rig parameters to optimize operations. The customer lacked granular insights during drilling, could not ascertain the root cause of non-productive time, and manual interpretation of signals led to missing of anomalies further degrading performance.
Complete Description
Cerebra product extracted and ingested different types of signals from surface and downhole sensors to perform near real-time processing. More than 170 vital signals every second from each oil rig were processed by Cerebra to provide near real time insights into drilling operations. This was achieved by handling Data Format and Data Extraction standards and Cerebra’s Visualization Studio provides the flexibility of generating customized asset utilization reports, thus helping the oilfield engineers to understand the root causes of non-productive time and better utilize the assets on field. Rig specific utilization reports, and weekly and monthly utilization reports helped to plan drilling operations improving drilling efficiency.
Ratio of potential issues which are "of interest" for human evaluation. Ideal target is to reduce the current volume by 80%.
Improve accuracy
1
Precision
Correctly Predicted Anomalous scenarios/ Total Anomalous scenarios predicted
1
Features related to adulterants in radio spectrum
Intensities around NIR range
1
Average vehicle driving speed
Average vehicle driving speed on all the road sections in a given region
Improve the road utilization efficiency
1
Coverage
Ratio of EMR QC requirements done in the solution/all issued EMR QC requirements in China. Ideal target is 100%.
Improve accuracy
1
Accuracy
The number of correctly recognized users’ intent over total number of users. Currently, accuracy reaches 95%.
Improve accuracy of recognizing users’ intent
1
Classifier Accuracy
Without straightening and pre-processing, the average classification accuracy obtained was 68.5%. However, with preprocessing, the classification accuracy improved to 86.7%. These results are very likely to improve with more annotated training data for classification.
1
Closeness to Golden Batch
How close a process is to the best possible batch
Helps in isolation of bad batches from good batches by identifying combination of process variable trajectories that lead to good or bad batch operation.
1
Model Accuracy
Accuracy of the prediction model
The extent to which the setpoints have correctly predicted
1
Ratio of ML discovered failure rate to nominal failure rate
What combination of manufacturing processes/decisions leads to higher failure rates compared to nominal failure rate
Actionable intelligence to improve the manufacturing process of HV circuit breakers
1
Number of labors
reduced % of labors
improvement of productivity
1
Customer Satisfaction
The ratio of customer satisfaction when using this system for requests. The expectation is 100%
Increasing its ratio as high as possible
1
MIoU (Mean Intersection over Union)
The intersection of prediction area and actual area divided by the union of the predicted area and the actual area. Ideal target is 100%.
Improve accuracy
1
Classification Ratio
Real to Pseudo wrong classification
Establishes the quality of identification
1
Ease of use
Simplicity and efficiency during initial learning. Teaching process should be easy.
1
Zone of Influence/ Thermal Correlation Index
Extent of influence of ACUs on data center racks.
Helps in improved control.
1
Invisible Loss Time
Indicates the lost time of the asset in being idle or off or unplanned downtime
Asset Utilization Reports indicate the effectively utilized time there indicating the lost time and their causes
1
Prediction Accuracy
To what extent has the model been able to predict correctly
Provided ability as to % of times the quality complied
1
Algorithm accuracy
Output when compared to the human expert analysis of the same data
See Reference
1
Generation of Activities (land use information and time of travel)
Purpose of activities is assigned based on land use information and time of travel. Census data and national/ local travel surveys will provide validation for the process
Phrase 1
1
accuracy
The accuracy of infraction and incident detection from traffic pictures/videos
To increase the accuracy of traffic monitoring and inspection
2
Split
Proportion of the potential issues which are "more likely to be a valid issue" for our end users.
Average vehicle waiting time at all the intersections in a given region
Improve the road utilization efficiency
2
Resolution
The number of answers solved over total number of questions asked
Improve the resolution of questions from users
2
Annotation Completeness
35.9 chromosomes segmented out after crowd annotation, for 50 images having 46 chromosomes
2
% Reduction in Calibration Time
The amount of time saved from manually setting the calibration
2
Number of complaints
reduced % of labor's complaint
improvement of productivity
2
Accuracy
Among all the predicted customer sentiment classification, the ratio of accurate prediction, current value is 76.4%
Increasing to 90%
2
FAR (false acceptance rate)
Negative samples are identified as positive samples / Total number of negative samples.The low FAR, the more smartphone will get correct scenes and objects
Improve accuracy
2
Training efficiency
Amount of necessary data for training might lead to practical obstacles in application.
2
Overall drilling time
The time spent on one drilling job inclusive of the all downtimes
Real Time visibility into operations gives the operations early warnings to take actions immediately.
2
Generation of agents (travel times, speed on link)
Agents generated will build up in the network creating realistic conditionsw of congestion. Speed on links.
Phrase 2
2
split
Proportion of images requiring human inspection. The less the split, the higher the efficiency.
To minimize the human effort in inspection
3
Satisfaction
The number of users who are satisfied with customer service over total number of users
Improve user experience
3
Lead time
time from order to shipment
improvement of productivity
3
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 90%
Increasing to 90%
3
Initial success rate
After initial training, the success rate needs to be acceptable such that the system can be put in the production line.
3
Opeartion of service (number of users for the service)
Optimisation of route and operation time in the day. Validation provided using data collected by Mobility service operators during the operation of service
Phrase 3
3
resource utilization ratio
Achievable resource utilization ratio in the hardware infrastructure ( the higher the utilization ratio, the lower amount the required resource)
To reduce the infrastructure investment and overall solution cost
4
Accuracy
Among all the predicted customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
4
Speed of improvement
Higher convergence speed of the reinforcement algorithm is making the solution more attractive.
5
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
5
Operational efficiency
Cycle time is the primary measure in manufacturing industry.
6
Success rate
Very high success rate is required for the solution to be accepted.
Mandate of the key sensors based on the type of equipment Based on the type of equipment, the makers need to have the basic set on sensors imbibed onto the system. E.g. for a pump – it is important to measure the input flow and output flow rates, vibrations, rotation speed, lube oil temperature and pressure. This will guide the equipment manufactures to provide their customers and their data products to capture the minimum required data and understand the equipment performance.
Mandate for the organizations to expose the minimum and key parameters. The equipment owners need to enable the basic set of sensors for the equipment health and performance which are required for monitoring the asset from any failures.
Standards for data formats Each organization has a different way of capturing data and storing them in different formats. Due to which the solutions are not scalable across organizations though the product behind them is same. It takes customised efforts each time.
Guidelines for deciding the sampling frequency based on the type of data We see a need to have a specific set of guidelines to capture data at a minimum required sampling frequency. For e.g. a vibration sensor should capture data at least at 1 ms.
Guidelines for feature engineering There must be guidelines as to how the features need to be engineered for AI models. Lack of this would lead to more black box models not explaining how the models behave the way they do.
Guidelines for standardization of event types and codes There are multiple events which occur for an asset or in a manufacturing plant. Guidelines would help people capture the data in a similar fashion helping the industry to benchmark against one another and at industry level we can understand, which events are the most critical.
Guidelines for standardization of fault and error codes for an equipment or process Similar to events, it is also useful to capture fault, failure and error codes in a standard way.
Process guidelines for event related data (maintenance and work orders) Guidelines would help people capture the data in a similar fashion helping the industry to benchmark against one another and at industry level we can understand, which events are the most critical.
Editor's comments and enhancements are shown in green. [✓ Reviewed]
The quality of use case submissions will be evaluated for inclusion in the Working Group's Technical Report based on the application area, relevant AI technologies, credible reference sources (see References section), and the following characteristics:
[1] Data Focus & Learning: Use cases for AI system which utilizes Machine Learning, and those that use a fixed a priori knowledge base.
[2] Level of Autonomy: Use cases demonstrating several degrees (dependent, autonomous, human/critic in the loop, etc.) of AI system autonomy.
[3] Verifiability & Transparency: Use cases demonstrating several types and levels of verifiability and transparency, including approaches for explainable AI, accountability, etc.
[4] Impact: Use cases demonstrating the impact of AI systems to society, environment, etc.
[5] Architecture: Use cases demonstrating several architectural paradigms for AI systems (e.g., cloud, distributed AI, crowdsourcing, swarm intelligence, etc.)
[6] Functional aspects, trustworthiness, and societal concerns
[7] AI life cycle components include acquire/process/apply.
These characteristics are identified in red in the use case.
Batch/Continuous/Discrete Manufacturing (Deployed in 75+ manufacturing lines in 10+ countries; Specifically identified the contributors to quality; predict potential quality failures).
Cerebra IOT signal intelligence platform provides the ability to have a holistic perspective and understanding of the sensitivity of the key parameters affecting output quality and ability to monitor and control the process in real-time. This will avoid variations in yields, build-up of inventories and missed customer deadlines.
Complete Description
Cerebra IOT signal intelligence platform ingested 3+ years of process data and sensor data regarding plant operations from temperature, rpm, torque and pressure sensors which were strapped on to industrial mixers. These are the mandatory sensors for the operations. Cerebra used its episode detection algorithms (deep learning) to filter signal from noise and specifically identify the contributors to quality (anomaly signatures) that can then be used as signals to predict quality. It used its proprietary N-dimensional Euclidian distance-based scoring algorithms to normalize and present a unified score to the business team. This unified health score provided the process team a different lens to benchmark, specifically target and radically improve process efficiencies. Cerebra then leveraged its sophisticated ensemble models to predict potential quality failures allowing the operations team to take real-time actions to control process deviations. The signals identified in the earlier steps provide Model Explainability to the end-user for reasons behind Quality deviation.
Challenges to accountability, New Security Threats.
Performance Indicators (KPIs)
Seq. No.
Name
Description
Reference to mentioned use case objectives
1
Coverage
Ratio of potential issues which are "of interest" for human evaluation. Ideal target is to reduce the current volume by 80%.
Improve accuracy
1
Precision
Correctly Predicted Anomalous scenarios/ Total Anomalous scenarios predicted
1
Features related to adulterants in radio spectrum
Intensities around NIR range
1
Average vehicle driving speed
Average vehicle driving speed on all the road sections in a given region
Improve the road utilization efficiency
1
Coverage
Ratio of EMR QC requirements done in the solution/all issued EMR QC requirements in China. Ideal target is 100%.
Improve accuracy
1
Accuracy
The number of correctly recognized users’ intent over total number of users. Currently, accuracy reaches 95%.
Improve accuracy of recognizing users’ intent
1
Classifier Accuracy
Without straightening and pre-processing, the average classification accuracy obtained was 68.5%. However, with preprocessing, the classification accuracy improved to 86.7%. These results are very likely to improve with more annotated training data for classification.
1
Closeness to Golden Batch
How close a process is to the best possible batch
Helps in isolation of bad batches from good batches by identifying combination of process variable trajectories that lead to good or bad batch operation.
1
Model Accuracy
Accuracy of the prediction model
The extent to which the setpoints have correctly predicted
1
Ratio of ML discovered failure rate to nominal failure rate
What combination of manufacturing processes/decisions leads to higher failure rates compared to nominal failure rate
Actionable intelligence to improve the manufacturing process of HV circuit breakers
1
Number of labors
reduced % of labors
improvement of productivity
1
Customer Satisfaction
The ratio of customer satisfaction when using this system for requests. The expectation is 100%
Increasing its ratio as high as possible
1
MIoU (Mean Intersection over Union)
The intersection of prediction area and actual area divided by the union of the predicted area and the actual area. Ideal target is 100%.
Improve accuracy
1
Classification Ratio
Real to Pseudo wrong classification
Establishes the quality of identification
1
Ease of use
Simplicity and efficiency during initial learning. Teaching process should be easy.
1
Zone of Influence/ Thermal Correlation Index
Extent of influence of ACUs on data center racks.
Helps in improved control.
1
Invisible Loss Time
Indicates the lost time of the asset in being idle or off or unplanned downtime
Asset Utilization Reports indicate the effectively utilized time there indicating the lost time and their causes
1
Prediction Accuracy
To what extent has the model been able to predict correctly
Provided ability as to % of times the quality complied
1
Algorithm accuracy
Output when compared to the human expert analysis of the same data
See Reference
1
Generation of Activities (land use information and time of travel)
Purpose of activities is assigned based on land use information and time of travel. Census data and national/ local travel surveys will provide validation for the process
Phrase 1
1
accuracy
The accuracy of infraction and incident detection from traffic pictures/videos
To increase the accuracy of traffic monitoring and inspection
2
Split
Proportion of the potential issues which are "more likely to be a valid issue" for our end users.
Average vehicle waiting time at all the intersections in a given region
Improve the road utilization efficiency
2
Resolution
The number of answers solved over total number of questions asked
Improve the resolution of questions from users
2
Annotation Completeness
35.9 chromosomes segmented out after crowd annotation, for 50 images having 46 chromosomes
2
% Reduction in Calibration Time
The amount of time saved from manually setting the calibration
2
Number of complaints
reduced % of labor's complaint
improvement of productivity
2
Accuracy
Among all the predicted customer sentiment classification, the ratio of accurate prediction, current value is 76.4%
Increasing to 90%
2
FAR (false acceptance rate)
Negative samples are identified as positive samples / Total number of negative samples.The low FAR, the more smartphone will get correct scenes and objects
Improve accuracy
2
Training efficiency
Amount of necessary data for training might lead to practical obstacles in application.
2
Overall drilling time
The time spent on one drilling job inclusive of the all downtimes
Real Time visibility into operations gives the operations early warnings to take actions immediately.
2
Generation of agents (travel times, speed on link)
Agents generated will build up in the network creating realistic conditionsw of congestion. Speed on links.
Phrase 2
2
split
Proportion of images requiring human inspection. The less the split, the higher the efficiency.
To minimize the human effort in inspection
3
Satisfaction
The number of users who are satisfied with customer service over total number of users
Improve user experience
3
Lead time
time from order to shipment
improvement of productivity
3
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 90%
Increasing to 90%
3
Initial success rate
After initial training, the success rate needs to be acceptable such that the system can be put in the production line.
3
Opeartion of service (number of users for the service)
Optimisation of route and operation time in the day. Validation provided using data collected by Mobility service operators during the operation of service
Phrase 3
3
resource utilization ratio
Achievable resource utilization ratio in the hardware infrastructure ( the higher the utilization ratio, the lower amount the required resource)
To reduce the infrastructure investment and overall solution cost
4
Accuracy
Among all the predicted customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
4
Speed of improvement
Higher convergence speed of the reinforcement algorithm is making the solution more attractive.
5
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
5
Operational efficiency
Cycle time is the primary measure in manufacturing industry.
6
Success rate
Very high success rate is required for the solution to be accepted.
Mandate of the key sensors based on the type of equipment. Based on the type of equipment, the makers need to have the basic set on sensors imbibed onto the system. e.g. for a pump – it is important to measure the input flow and output flow rates, vibrations, rotation speed, lube oil temperature and pressure. This will guide the equipment manufactures to provide their customers and their data products to capture the minimum required data and understand the equipment performance.
Mandate for the organizations to expose the minimum and key parameters. The equipment owners need to enable the basic set of sensors for the equipment health and performance which are required for monitoring the asset from any failures.
Standards for Data Formats Each organization has a different way of capturing data and storing them in different formats. Due to this, the solutions are not scalable across organizations though the product behind them is same. It takes customised efforts each time.
Guidelines for deciding the sampling frequency based on the type of data. We see a need to have a specific set of guidelines to capture data at a minimum required sampling frequency, e.g. a vibration sensor should capture data at least at 1 ms or less.
Guidelines for Feature Engineering. There must be guidelines as to how the features need to be engineered for AI models. Lack of this would lead to more black box models not explaining how the models behave the way they do.
Guidelines for Standardization of event types and codes. There are multiple events which occur for an asset or in a manufacturing plant. Guidelines would help people capture the data in a similar fashion helping the industry to benchmark against one another and at industry level we can understand, which events are the most critical.
Guidelines for standardization of Fault and Error Codes for an equipment or process. Similar to events, it is also useful to capture fault, failure and error codes in a standard way.
Process Guidelines for event related data (Maintenance and Work Orders): Guidelines would help people capture the data in a similar fashion helping the industry to benchmark against one another and at industry level we can understand, which events are the most critical.
Guidelines for Training AI models: A defined set of guidelines for AI models would be useful for the data scientists to follow. It will also aid the consumers of AI models to understand how the outcome has been deduced.
Editor's comments and enhancements are shown in green. [✓ Reviewed]
The quality of use case submissions will be evaluated for inclusion in the Working Group's Technical Report based on the application area, relevant AI technologies, credible reference sources (see References section), and the following characteristics:
[1] Data Focus & Learning: Use cases for AI system which utilizes Machine Learning, and those that use a fixed a priori knowledge base.
[2] Level of Autonomy: Use cases demonstrating several degrees (dependent, autonomous, human/critic in the loop, etc.) of AI system autonomy.
[3] Verifiability & Transparency: Use cases demonstrating several types and levels of verifiability and transparency, including approaches for explainable AI, accountability, etc.
[4] Impact: Use cases demonstrating the impact of AI systems to society, environment, etc.
[5] Architecture: Use cases demonstrating several architectural paradigms for AI systems (e.g., cloud, distributed AI, crowdsourcing, swarm intelligence, etc.)
[6] Functional aspects, trustworthiness, and societal concerns
[7] AI life cycle components include acquire/process/apply.
These characteristics are identified in red in the use case.
No.
51
ID:
Use Case Name:
Machine Learning Tools in Support of Transformer Diagnostics
Use of Machine Learning (ML) algorithms as supporting tools for the automatic classification of power transformers operating condition
Short Description (up to 150 words)
The successful use of ML tools may find multiple applications in the industry such as providing fast ways of analysing new data streaming from online sensors, evaluating the importance of individual variables in the context of transformer condition assessment and also the need or adequacy of data imputation in the so widely common problem of missing data
Complete Description
The work consists of training 12 ML algorithms with real data from 1,000 (one thousand) transformers that were individually analyzed by human experts. Each transformer in the database is scored with a ‘green’, ‘yellow’ or ‘red’ card depending on the data, the interpretation of human experts, or even after some calculations carried out by the company’s internal algorithms frequently utilized by the experts to identify units with technical operational issues. The ML algorithms, however, do not utilize or are given any of the engineering tools employed by the human experts. The algorithms only employed the raw data in a supervised learning process in which a column named ‘Class’ was added to the transformer information with the classification red, yellow or green provided by the human expert.
Ratio of potential issues which are "of interest" for human evaluation. Ideal target is to reduce the current volume by 80%.
Improve accuracy
1
Precision
Correctly Predicted Anomalous scenarios/ Total Anomalous scenarios predicted
1
Features related to adulterants in radio spectrum
Intensities around NIR range
1
Average vehicle driving speed
Average vehicle driving speed on all the road sections in a given region
Improve the road utilization efficiency
1
Coverage
Ratio of EMR QC requirements done in the solution/all issued EMR QC requirements in China. Ideal target is 100%.
Improve accuracy
1
Accuracy
The number of correctly recognized users’ intent over total number of users. Currently, accuracy reaches 95%.
Improve accuracy of recognizing users’ intent
1
Classifier Accuracy
Without straightening and pre-processing, the average classification accuracy obtained was 68.5%. However, with preprocessing, the classification accuracy improved to 86.7%. These results are very likely to improve with more annotated training data for classification.
1
Closeness to Golden Batch
How close a process is to the best possible batch
Helps in isolation of bad batches from good batches by identifying combination of process variable trajectories that lead to good or bad batch operation.
1
Model Accuracy
Accuracy of the prediction model
The extent to which the setpoints have correctly predicted
1
Ratio of ML discovered failure rate to nominal failure rate
What combination of manufacturing processes/decisions leads to higher failure rates compared to nominal failure rate
Actionable intelligence to improve the manufacturing process of HV circuit breakers
1
Number of labors
reduced % of labors
improvement of productivity
1
Customer Satisfaction
The ratio of customer satisfaction when using this system for requests. The expectation is 100%
Increasing its ratio as high as possible
1
MIoU (Mean Intersection over Union)
The intersection of prediction area and actual area divided by the union of the predicted area and the actual area. Ideal target is 100%.
Improve accuracy
1
Classification Ratio
Real to Pseudo wrong classification
Establishes the quality of identification
1
Ease of use
Simplicity and efficiency during initial learning. Teaching process should be easy.
1
Zone of Influence/ Thermal Correlation Index
Extent of influence of ACUs on data center racks.
Helps in improved control.
1
Invisible Loss Time
Indicates the lost time of the asset in being idle or off or unplanned downtime
Asset Utilization Reports indicate the effectively utilized time there indicating the lost time and their causes
1
Prediction Accuracy
To what extent has the model been able to predict correctly
Provided ability as to % of times the quality complied
1
Algorithm accuracy
Output when compared to the human expert analysis of the same data
See Reference
1
Generation of Activities (land use information and time of travel)
Purpose of activities is assigned based on land use information and time of travel. Census data and national/ local travel surveys will provide validation for the process
Phrase 1
1
accuracy
The accuracy of infraction and incident detection from traffic pictures/videos
To increase the accuracy of traffic monitoring and inspection
2
Split
Proportion of the potential issues which are "more likely to be a valid issue" for our end users.
Average vehicle waiting time at all the intersections in a given region
Improve the road utilization efficiency
2
Resolution
The number of answers solved over total number of questions asked
Improve the resolution of questions from users
2
Annotation Completeness
35.9 chromosomes segmented out after crowd annotation, for 50 images having 46 chromosomes
2
% Reduction in Calibration Time
The amount of time saved from manually setting the calibration
2
Number of complaints
reduced % of labor's complaint
improvement of productivity
2
Accuracy
Among all the predicted customer sentiment classification, the ratio of accurate prediction, current value is 76.4%
Increasing to 90%
2
FAR (false acceptance rate)
Negative samples are identified as positive samples / Total number of negative samples.The low FAR, the more smartphone will get correct scenes and objects
Improve accuracy
2
Training efficiency
Amount of necessary data for training might lead to practical obstacles in application.
2
Overall drilling time
The time spent on one drilling job inclusive of the all downtimes
Real Time visibility into operations gives the operations early warnings to take actions immediately.
2
Generation of agents (travel times, speed on link)
Agents generated will build up in the network creating realistic conditionsw of congestion. Speed on links.
Phrase 2
2
split
Proportion of images requiring human inspection. The less the split, the higher the efficiency.
To minimize the human effort in inspection
3
Satisfaction
The number of users who are satisfied with customer service over total number of users
Improve user experience
3
Lead time
time from order to shipment
improvement of productivity
3
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 90%
Increasing to 90%
3
Initial success rate
After initial training, the success rate needs to be acceptable such that the system can be put in the production line.
3
Opeartion of service (number of users for the service)
Optimisation of route and operation time in the day. Validation provided using data collected by Mobility service operators during the operation of service
Phrase 3
3
resource utilization ratio
Achievable resource utilization ratio in the hardware infrastructure ( the higher the utilization ratio, the lower amount the required resource)
To reduce the infrastructure investment and overall solution cost
4
Accuracy
Among all the predicted customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
4
Speed of improvement
Higher convergence speed of the reinforcement algorithm is making the solution more attractive.
5
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
5
Operational efficiency
Cycle time is the primary measure in manufacturing industry.
6
Success rate
Very high success rate is required for the solution to be accepted.
Editor's comments and enhancements are shown in green. [✓ Reviewed]
The quality of use case submissions will be evaluated for inclusion in the Working Group's Technical Report based on the application area, relevant AI technologies, credible reference sources (see References section), and the following characteristics:
[1] Data Focus & Learning: Use cases for AI system which utilizes Machine Learning, and those that use a fixed a priori knowledge base.
[2] Level of Autonomy: Use cases demonstrating several degrees (dependent, autonomous, human/critic in the loop, etc.) of AI system autonomy.
[3] Verifiability & Transparency: Use cases demonstrating several types and levels of verifiability and transparency, including approaches for explainable AI, accountability, etc.
[4] Impact: Use cases demonstrating the impact of AI systems to society, environment, etc.
[5] Architecture: Use cases demonstrating several architectural paradigms for AI systems (e.g., cloud, distributed AI, crowdsourcing, swarm intelligence, etc.)
[6] Functional aspects, trustworthiness, and societal concerns
[7] AI life cycle components include acquire/process/apply.
These characteristics are identified in red in the use case.
No.
52
ID:
Use Case Name:
Automated Travel Pattern Recognition using Mobile Network Data for Applications to Mobility as a Service
Application Domain
Transportation
Deployment Model
Activity- based Modelling for New mobility Services
Phase 1: Attribute trip purpose and mode of transport to multimodal door-to-door journeys from Mobile phone Network Dataset using AI and machine learning techniques (Activity based model) Phase 2: Generate daily activities for static agents in the Agent Based Model Phase 3: Optimisation of New Mobility services in integration with mass transit
Short Description (up to 150 words)
Activity- based modelling has the capability to exploit big data source generated by smart cities to create a digital twin of urban environments to test Mobility as a Service schemes. MND data have been used to create activities for an Agent Based Model. AI is used to automatically detect purpose and mode of transport in multimodal round trips, obtained by anonymized and aggregated MND trip-chains dataset. Data fusion techniques and SQL queries were also used to consider land use and facilities in the urban area of interest.
Complete Description
Activity- based modelling has the capability to exploit big data source generated by smart cities to create a digital twin of urban environments to test Mobility as a Service schemes. Given the rise of location- based data and Mobile phone Network Data (MND) for transport modelling purpose, Agent based modelling has become a viable tool to explore a sustainable introduction of mobility services, exploring the integration with mass transit. AI is used in detecting purpose and mode of transport in multimodal round trips and assign purpose and mode of transport to trip- chains dataset coming from MND. The methodology has been developed for the Innovate UK funded Mobility on Demand Laboratory Environment (MODLE) project and will undergo a validation process during the Demand Modelling and Assessment through a Network Demonstrator (DeMAND) project for the Department for Transport (UK)
Ratio of potential issues which are "of interest" for human evaluation. Ideal target is to reduce the current volume by 80%.
Improve accuracy
1
Precision
Correctly Predicted Anomalous scenarios/ Total Anomalous scenarios predicted
1
Features related to adulterants in radio spectrum
Intensities around NIR range
1
Average vehicle driving speed
Average vehicle driving speed on all the road sections in a given region
Improve the road utilization efficiency
1
Coverage
Ratio of EMR QC requirements done in the solution/all issued EMR QC requirements in China. Ideal target is 100%.
Improve accuracy
1
Accuracy
The number of correctly recognized users’ intent over total number of users. Currently, accuracy reaches 95%.
Improve accuracy of recognizing users’ intent
1
Classifier Accuracy
Without straightening and pre-processing, the average classification accuracy obtained was 68.5%. However, with preprocessing, the classification accuracy improved to 86.7%. These results are very likely to improve with more annotated training data for classification.
1
Closeness to Golden Batch
How close a process is to the best possible batch
Helps in isolation of bad batches from good batches by identifying combination of process variable trajectories that lead to good or bad batch operation.
1
Model Accuracy
Accuracy of the prediction model
The extent to which the setpoints have correctly predicted
1
Ratio of ML discovered failure rate to nominal failure rate
What combination of manufacturing processes/decisions leads to higher failure rates compared to nominal failure rate
Actionable intelligence to improve the manufacturing process of HV circuit breakers
1
Number of labors
reduced % of labors
improvement of productivity
1
Customer Satisfaction
The ratio of customer satisfaction when using this system for requests. The expectation is 100%
Increasing its ratio as high as possible
1
MIoU (Mean Intersection over Union)
The intersection of prediction area and actual area divided by the union of the predicted area and the actual area. Ideal target is 100%.
Improve accuracy
1
Classification Ratio
Real to Pseudo wrong classification
Establishes the quality of identification
1
Ease of use
Simplicity and efficiency during initial learning. Teaching process should be easy.
1
Zone of Influence/ Thermal Correlation Index
Extent of influence of ACUs on data center racks.
Helps in improved control.
1
Invisible Loss Time
Indicates the lost time of the asset in being idle or off or unplanned downtime
Asset Utilization Reports indicate the effectively utilized time there indicating the lost time and their causes
1
Prediction Accuracy
To what extent has the model been able to predict correctly
Provided ability as to % of times the quality complied
1
Algorithm accuracy
Output when compared to the human expert analysis of the same data
See Reference
1
Generation of Activities (land use information and time of travel)
Purpose of activities is assigned based on land use information and time of travel. Census data and national/ local travel surveys will provide validation for the process
Phrase 1
1
accuracy
The accuracy of infraction and incident detection from traffic pictures/videos
To increase the accuracy of traffic monitoring and inspection
2
Split
Proportion of the potential issues which are "more likely to be a valid issue" for our end users.
Average vehicle waiting time at all the intersections in a given region
Improve the road utilization efficiency
2
Resolution
The number of answers solved over total number of questions asked
Improve the resolution of questions from users
2
Annotation Completeness
35.9 chromosomes segmented out after crowd annotation, for 50 images having 46 chromosomes
2
% Reduction in Calibration Time
The amount of time saved from manually setting the calibration
2
Number of complaints
reduced % of labor's complaint
improvement of productivity
2
Accuracy
Among all the predicted customer sentiment classification, the ratio of accurate prediction, current value is 76.4%
Increasing to 90%
2
FAR (false acceptance rate)
Negative samples are identified as positive samples / Total number of negative samples.The low FAR, the more smartphone will get correct scenes and objects
Improve accuracy
2
Training efficiency
Amount of necessary data for training might lead to practical obstacles in application.
2
Overall drilling time
The time spent on one drilling job inclusive of the all downtimes
Real Time visibility into operations gives the operations early warnings to take actions immediately.
2
Generation of agents (travel times, speed on link)
Agents generated will build up in the network creating realistic conditionsw of congestion. Speed on links.
Phrase 2
2
split
Proportion of images requiring human inspection. The less the split, the higher the efficiency.
To minimize the human effort in inspection
3
Satisfaction
The number of users who are satisfied with customer service over total number of users
Improve user experience
3
Lead time
time from order to shipment
improvement of productivity
3
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 90%
Increasing to 90%
3
Initial success rate
After initial training, the success rate needs to be acceptable such that the system can be put in the production line.
3
Opeartion of service (number of users for the service)
Optimisation of route and operation time in the day. Validation provided using data collected by Mobility service operators during the operation of service
Phrase 3
3
resource utilization ratio
Achievable resource utilization ratio in the hardware infrastructure ( the higher the utilization ratio, the lower amount the required resource)
To reduce the infrastructure investment and overall solution cost
4
Accuracy
Among all the predicted customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
4
Speed of improvement
Higher convergence speed of the reinforcement algorithm is making the solution more attractive.
5
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
5
Operational efficiency
Cycle time is the primary measure in manufacturing industry.
6
Success rate
Very high success rate is required for the solution to be accepted.
AI Features
Task(s)
Assign purpose of each trip in the chain, assign model of transport for each trip in the chain, generate daily activity plans, generate static agents (users), generate dynamic agents (service)
Editor's comments and enhancements are shown in green. [✓ Reviewed]
The quality of use case submissions will be evaluated for inclusion in the Working Group's Technical Report based on the application area, relevant AI technologies, credible reference sources (see References section), and the following characteristics:
[1] Data Focus & Learning: Use cases for AI system which utilizes Machine Learning, and those that use a fixed a priori knowledge base.
[2] Level of Autonomy: Use cases demonstrating several degrees (dependent, autonomous, human/critic in the loop, etc.) of AI system autonomy.
[3] Verifiability & Transparency: Use cases demonstrating several types and levels of verifiability and transparency, including approaches for explainable AI, accountability, etc.
[4] Impact: Use cases demonstrating the impact of AI systems to society, environment, etc.
[5] Architecture: Use cases demonstrating several architectural paradigms for AI systems (e.g., cloud, distributed AI, crowdsourcing, swarm intelligence, etc.)
[6] Functional aspects, trustworthiness, and societal concerns
[7] AI life cycle components include acquire/process/apply.
These characteristics are identified in red in the use case.
No.
29
ID:
Use Case Name:
Enhancing traffic management efficiency and infraction detection accuracy with AI technologies
To increase the accuracy and efficiency of infraction detection, traffic monitoring and flow analysis, while minimizing the human effort and the overall solution cost.
Short Description (up to 150 words)
Big data enabled AI technologies are applied to monitoring and managing the traffic in a large municipality in China. Multi-sourced data (traffic flow, vehicle data, pedestrian movement, etc.) is monitored, from which illegal operation of vehicles, unexpected incidents, surge of traffic etc. are detected and analysed with machine learning (ML) methods. ML tasks (including training and deployment) are carried out on a platform supporting the integration of various ML frameworks, models and algorithms. The platform is based on heterogeneous computing resources. The efficiency and accuracy of infraction detection, and the effectiveness of traffic management are significantly improved, with much reduced human effort and overall solution cost.
Complete Description
With the population and the number of vehicles growing in large cities, managing the heavy traffic in urban areas has become a challenging yet essential task for the municipality. Addressing this issue has become particularly urgent for big cities in China, where millions of people live and commute every day. In this use case, big data based AI technologies are applied to monitoring and managing the heavy traffic in a metropolitan in south China. Previously, significant human resources were involved in the vehicle and road monitoring, and large investment was made to the computing infrastructure specific to certain functionalities. To increase the efficiency of urban transportation, reduce the traffic jam and air pollution, as well as minimize the human effort, machine learning techniques (e.g. deep learning) are applied to image and video analysis, such as traffic flow analysis, infraction detection and incident detection. Example applications include but not limited to 1) detection of traffic rule violation, e.g. over-speeding, wrong driving lanes or parking. AI-enabled detection produces much faster and more accurate result, and helps in enforcing the traffic regulation. 2) traffic light optimization. Based on the modelling and analysis of multi-sourced traffic information (both real-time and historical data), traffic lights are dynamically configured to divert the flow, increase the passing speed of cars and reduce the traffic jam in major junctions. The use of AI has obtained remarkable results: The infraction detection efficiency gets 10X increase, and the detection accuracy is greater than 95%. The urban area traffic jam is much alleviated, with vehicles’ passing speed through major junctions increases by 9%-25%.
Ratio of potential issues which are "of interest" for human evaluation. Ideal target is to reduce the current volume by 80%.
Improve accuracy
1
Precision
Correctly Predicted Anomalous scenarios/ Total Anomalous scenarios predicted
1
Features related to adulterants in radio spectrum
Intensities around NIR range
1
Average vehicle driving speed
Average vehicle driving speed on all the road sections in a given region
Improve the road utilization efficiency
1
Coverage
Ratio of EMR QC requirements done in the solution/all issued EMR QC requirements in China. Ideal target is 100%.
Improve accuracy
1
Accuracy
The number of correctly recognized users’ intent over total number of users. Currently, accuracy reaches 95%.
Improve accuracy of recognizing users’ intent
1
Classifier Accuracy
Without straightening and pre-processing, the average classification accuracy obtained was 68.5%. However, with preprocessing, the classification accuracy improved to 86.7%. These results are very likely to improve with more annotated training data for classification.
1
Closeness to Golden Batch
How close a process is to the best possible batch
Helps in isolation of bad batches from good batches by identifying combination of process variable trajectories that lead to good or bad batch operation.
1
Model Accuracy
Accuracy of the prediction model
The extent to which the setpoints have correctly predicted
1
Ratio of ML discovered failure rate to nominal failure rate
What combination of manufacturing processes/decisions leads to higher failure rates compared to nominal failure rate
Actionable intelligence to improve the manufacturing process of HV circuit breakers
1
Number of labors
reduced % of labors
improvement of productivity
1
Customer Satisfaction
The ratio of customer satisfaction when using this system for requests. The expectation is 100%
Increasing its ratio as high as possible
1
MIoU (Mean Intersection over Union)
The intersection of prediction area and actual area divided by the union of the predicted area and the actual area. Ideal target is 100%.
Improve accuracy
1
Classification Ratio
Real to Pseudo wrong classification
Establishes the quality of identification
1
Ease of use
Simplicity and efficiency during initial learning. Teaching process should be easy.
1
Zone of Influence/ Thermal Correlation Index
Extent of influence of ACUs on data center racks.
Helps in improved control.
1
Invisible Loss Time
Indicates the lost time of the asset in being idle or off or unplanned downtime
Asset Utilization Reports indicate the effectively utilized time there indicating the lost time and their causes
1
Prediction Accuracy
To what extent has the model been able to predict correctly
Provided ability as to % of times the quality complied
1
Algorithm accuracy
Output when compared to the human expert analysis of the same data
See Reference
1
Generation of Activities (land use information and time of travel)
Purpose of activities is assigned based on land use information and time of travel. Census data and national/ local travel surveys will provide validation for the process
Phrase 1
1
accuracy
The accuracy of infraction and incident detection from traffic pictures/videos
To increase the accuracy of traffic monitoring and inspection
2
Split
Proportion of the potential issues which are "more likely to be a valid issue" for our end users.
Average vehicle waiting time at all the intersections in a given region
Improve the road utilization efficiency
2
Resolution
The number of answers solved over total number of questions asked
Improve the resolution of questions from users
2
Annotation Completeness
35.9 chromosomes segmented out after crowd annotation, for 50 images having 46 chromosomes
2
% Reduction in Calibration Time
The amount of time saved from manually setting the calibration
2
Number of complaints
reduced % of labor's complaint
improvement of productivity
2
Accuracy
Among all the predicted customer sentiment classification, the ratio of accurate prediction, current value is 76.4%
Increasing to 90%
2
FAR (false acceptance rate)
Negative samples are identified as positive samples / Total number of negative samples.The low FAR, the more smartphone will get correct scenes and objects
Improve accuracy
2
Training efficiency
Amount of necessary data for training might lead to practical obstacles in application.
2
Overall drilling time
The time spent on one drilling job inclusive of the all downtimes
Real Time visibility into operations gives the operations early warnings to take actions immediately.
2
Generation of agents (travel times, speed on link)
Agents generated will build up in the network creating realistic conditionsw of congestion. Speed on links.
Phrase 2
2
split
Proportion of images requiring human inspection. The less the split, the higher the efficiency.
To minimize the human effort in inspection
3
Satisfaction
The number of users who are satisfied with customer service over total number of users
Improve user experience
3
Lead time
time from order to shipment
improvement of productivity
3
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 90%
Increasing to 90%
3
Initial success rate
After initial training, the success rate needs to be acceptable such that the system can be put in the production line.
3
Opeartion of service (number of users for the service)
Optimisation of route and operation time in the day. Validation provided using data collected by Mobility service operators during the operation of service
Phrase 3
3
resource utilization ratio
Achievable resource utilization ratio in the hardware infrastructure ( the higher the utilization ratio, the lower amount the required resource)
To reduce the infrastructure investment and overall solution cost
4
Accuracy
Among all the predicted customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
4
Speed of improvement
Higher convergence speed of the reinforcement algorithm is making the solution more attractive.
5
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
5
Operational efficiency
Cycle time is the primary measure in manufacturing industry.
6
Success rate
Very high success rate is required for the solution to be accepted.
Requirement of computing infrastructure to empower AI applications in the transportation domain, e.g. the integration of acceleration units (GPU, FPGA, etc.), dynamic scheduling and on-demand allocation of heterogeneous resources
Support of mainstream ML frameworks, and the algorithms and models from different vendors, to prevent vendor lock-in
Challenges & Issues
Constant improvement in hardware architecture to increase the performance and efficiency of running ML/DL tasks
Consistent interfaces between applications, ML engines and heterogeneous resource pools
Support of new models and emerging algorithms for growing functionalities
AI’s application in urban transportation significantly improves the quality of life for urban citizens, reduces the time wasted in heavy traffic and the air pollution from vehicles.
Vary (depending on the weather condition, environment etc.)
Editor's comments and enhancements are shown in green. [✓ Reviewed]
The quality of use case submissions will be evaluated for inclusion in the Working Group's Technical Report based on the application area, relevant AI technologies, credible reference sources (see References section), and the following characteristics:
[1] Data Focus & Learning: Use cases for AI system which utilizes Machine Learning, and those that use a fixed a priori knowledge base.
[2] Level of Autonomy: Use cases demonstrating several degrees (dependent, autonomous, human/critic in the loop, etc.) of AI system autonomy.
[3] Verifiability & Transparency: Use cases demonstrating several types and levels of verifiability and transparency, including approaches for explainable AI, accountability, etc.
[4] Impact: Use cases demonstrating the impact of AI systems to society, environment, etc.
[5] Architecture: Use cases demonstrating several architectural paradigms for AI systems (e.g., cloud, distributed AI, crowdsourcing, swarm intelligence, etc.)
[6] Functional aspects, trustworthiness, and societal concerns
[7] AI life cycle components include acquire/process/apply.
These characteristics are identified in red in the use case.
No.
30
ID:
Use Case Name:
Autonomous network and automation level definition
To define autonomous network concept and automation level for the common understanding and consensus
Short Description (up to 150 words)
With the goal of providing common understanding and consensus for autonomous self-driving network, this use case delivers a harmonized classification system and supporting definitions that:
Define the concept of autonomous network
Identify six levels of network automation from “no automation” to “full automation”.
Base definitions and levels on functional aspects of technology.
Describe categorical distinctions for a step-wise progression through the levels.
Educate a wider community by clarifying for each level what role (if any) operators have in performing the dynamic network operations task while a network automation system is engaged.
Complete Description
The telecom CSPs have a dual challenge – to increase agility while reducing network operating cost.
The exponential growth of network complexity e.g. 5G will make the traditional network O&M model unsustainable;
Digital transformation accelerates service innovation but requires automation capabilities.
As CSPs start to evaluate their digital transformation strategies, automation is a central concern. Some operators are already introducing automation to some of their network processes, most commonly O&M, planning and optimization. According to Analysys Mason, in 2018, 56% of CSPs globally have little or no automation in their networks. But by 2025, according to their own predictions, almost 80% expect to have automated 40% or more of their network operations, and one-third will have automated over 80%. The introduction of AI/ML (artificial intelligence/machine learning) will be an important part of that process for many CSPs, helping to make the network more intelligent, agile and predictive. The autonomous self-driving network has two essential elements in common with the autonomous self-driving car:
There are different levels of automation, relating to different timescales and scenarios
Intensive use of artificial intelligence (AI) is essential
With the goal of providing common understanding and consensus for autonomous self driving network, this use case delivers a harmonized classification system and supporting definitions that set out six levels of automation for the network.
Level
Name
Definition
Execution
(Hands)
Awareness
(Eyes)
Decision
(Minds)
Experience
(Hearts)
System
Capability
0
Manual
Operation
& Maintenance
Even
with auxiliary tools, O&M personnel perform all dynamic tasks.
P
P
P
P
n/a
1
Assisted
Operation
& Maintenance
Under
the applicable design scope, the system can execute a sub-task repeatedly
based on rules.
P/S
P
P
P
Sub-task
level
2
Partial
Autonomous
Network
Under
the applicable design scope, the system continuously completes the control
task of a unit based on the model.
S
P
P
P
Unit
level
3
Conditional
Autonomous
Network
Under
the applicable design scope, the system can implement complete closed-loop
automation of single-domain scenarios. Users can respond to the requests in a
timely manner when the system fails.
S
S
P
P
Domain
level
4
Highly
Autonomous
Network
Under
the applicable design scope, the system can automatically analyze and execute
cross-domain and service close-loop automation.
S
S
P
P
Service
level
5
Full
Autonomous
Network
The
system can perform complete dynamic tasks and exception handling in all
network environments. O&M personnel do not need to intervene.
S
S
S
P/S
All
Modes
P=Personnel (Manual), S=System (Automated) -Level 0 - manual O&M: The system delivers assisted monitoring capabilities, which means all dynamic tasks have to be executed manually. -Level 1 - assisted O&M: The system executes a certain sub-task based on existing rules to increase execution efficiency. -Level 2 - partial autonomous network: The system enables closed-loop O&M for certain units under certain external environments, lowering the bar for personnel experience and skills. -Level 3 - conditional autonomous network: Building on L2 capabilities, the system can sense real-time environmental changes, and in certain domains, optimize and adjust itself to the external environment to enable intent-based closed-loop management. -Level 4 - highly autonomous network: Building on L3 capabilities, the system enables, in a more complicated cross-domain environment, predictive or active closed-loop management of service and customer experience-driven networks. This allows operators to resolve network faults prior to customer complaints, reduce service outages and customer complaints, and ultimately, improve customer satisfaction. -Level 5 - full autonomous network: This level is the ultimate goal for telecom network evolution. The system possesses closed-loop automation capabilities across multiple services, multiple domains, and the entire lifecycle, achieving autonomous driving networks.
The lower levels can be applied now and deliver immediate cost and agility benefits in certain scenarios. An operator can then evolve to the higher levels, gaining additional benefits and addressing a wider range of scenarios. Network automation is a long run objective with step-to-step process, from providing an alternative to repetitive execution actions, to performing perception and monitoring of network environment and network device status, making decisions based on multiple factors and policies, and providing effective perception of end user experience. The system capability also starts from some service scenarios and covers all service scenarios.
Ratio of potential issues which are "of interest" for human evaluation. Ideal target is to reduce the current volume by 80%.
Improve accuracy
1
Precision
Correctly Predicted Anomalous scenarios/ Total Anomalous scenarios predicted
1
Features related to adulterants in radio spectrum
Intensities around NIR range
1
Average vehicle driving speed
Average vehicle driving speed on all the road sections in a given region
Improve the road utilization efficiency
1
Coverage
Ratio of EMR QC requirements done in the solution/all issued EMR QC requirements in China. Ideal target is 100%.
Improve accuracy
1
Accuracy
The number of correctly recognized users’ intent over total number of users. Currently, accuracy reaches 95%.
Improve accuracy of recognizing users’ intent
1
Classifier Accuracy
Without straightening and pre-processing, the average classification accuracy obtained was 68.5%. However, with preprocessing, the classification accuracy improved to 86.7%. These results are very likely to improve with more annotated training data for classification.
1
Closeness to Golden Batch
How close a process is to the best possible batch
Helps in isolation of bad batches from good batches by identifying combination of process variable trajectories that lead to good or bad batch operation.
1
Model Accuracy
Accuracy of the prediction model
The extent to which the setpoints have correctly predicted
1
Ratio of ML discovered failure rate to nominal failure rate
What combination of manufacturing processes/decisions leads to higher failure rates compared to nominal failure rate
Actionable intelligence to improve the manufacturing process of HV circuit breakers
1
Number of labors
reduced % of labors
improvement of productivity
1
Customer Satisfaction
The ratio of customer satisfaction when using this system for requests. The expectation is 100%
Increasing its ratio as high as possible
1
MIoU (Mean Intersection over Union)
The intersection of prediction area and actual area divided by the union of the predicted area and the actual area. Ideal target is 100%.
Improve accuracy
1
Classification Ratio
Real to Pseudo wrong classification
Establishes the quality of identification
1
Ease of use
Simplicity and efficiency during initial learning. Teaching process should be easy.
1
Zone of Influence/ Thermal Correlation Index
Extent of influence of ACUs on data center racks.
Helps in improved control.
1
Invisible Loss Time
Indicates the lost time of the asset in being idle or off or unplanned downtime
Asset Utilization Reports indicate the effectively utilized time there indicating the lost time and their causes
1
Prediction Accuracy
To what extent has the model been able to predict correctly
Provided ability as to % of times the quality complied
1
Algorithm accuracy
Output when compared to the human expert analysis of the same data
See Reference
1
Generation of Activities (land use information and time of travel)
Purpose of activities is assigned based on land use information and time of travel. Census data and national/ local travel surveys will provide validation for the process
Phrase 1
1
accuracy
The accuracy of infraction and incident detection from traffic pictures/videos
To increase the accuracy of traffic monitoring and inspection
2
Split
Proportion of the potential issues which are "more likely to be a valid issue" for our end users.
Average vehicle waiting time at all the intersections in a given region
Improve the road utilization efficiency
2
Resolution
The number of answers solved over total number of questions asked
Improve the resolution of questions from users
2
Annotation Completeness
35.9 chromosomes segmented out after crowd annotation, for 50 images having 46 chromosomes
2
% Reduction in Calibration Time
The amount of time saved from manually setting the calibration
2
Number of complaints
reduced % of labor's complaint
improvement of productivity
2
Accuracy
Among all the predicted customer sentiment classification, the ratio of accurate prediction, current value is 76.4%
Increasing to 90%
2
FAR (false acceptance rate)
Negative samples are identified as positive samples / Total number of negative samples.The low FAR, the more smartphone will get correct scenes and objects
Improve accuracy
2
Training efficiency
Amount of necessary data for training might lead to practical obstacles in application.
2
Overall drilling time
The time spent on one drilling job inclusive of the all downtimes
Real Time visibility into operations gives the operations early warnings to take actions immediately.
2
Generation of agents (travel times, speed on link)
Agents generated will build up in the network creating realistic conditionsw of congestion. Speed on links.
Phrase 2
2
split
Proportion of images requiring human inspection. The less the split, the higher the efficiency.
To minimize the human effort in inspection
3
Satisfaction
The number of users who are satisfied with customer service over total number of users
Improve user experience
3
Lead time
time from order to shipment
improvement of productivity
3
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 90%
Increasing to 90%
3
Initial success rate
After initial training, the success rate needs to be acceptable such that the system can be put in the production line.
3
Opeartion of service (number of users for the service)
Optimisation of route and operation time in the day. Validation provided using data collected by Mobility service operators during the operation of service
Phrase 3
3
resource utilization ratio
Achievable resource utilization ratio in the hardware infrastructure ( the higher the utilization ratio, the lower amount the required resource)
To reduce the infrastructure investment and overall solution cost
4
Accuracy
Among all the predicted customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
4
Speed of improvement
Higher convergence speed of the reinforcement algorithm is making the solution more attractive.
5
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
5
Operational efficiency
Cycle time is the primary measure in manufacturing industry.
6
Success rate
Very high success rate is required for the solution to be accepted.
Editor's comments and enhancements are shown in green. [✓ Reviewed]
The quality of use case submissions will be evaluated for inclusion in the Working Group's Technical Report based on the application area, relevant AI technologies, credible reference sources (see References section), and the following characteristics:
[1] Data Focus & Learning: Use cases for AI system which utilizes Machine Learning, and those that use a fixed a priori knowledge base.
[2] Level of Autonomy: Use cases demonstrating several degrees (dependent, autonomous, human/critic in the loop, etc.) of AI system autonomy.
[3] Verifiability & Transparency: Use cases demonstrating several types and levels of verifiability and transparency, including approaches for explainable AI, accountability, etc.
[4] Impact: Use cases demonstrating the impact of AI systems to society, environment, etc.
[5] Architecture: Use cases demonstrating several architectural paradigms for AI systems (e.g., cloud, distributed AI, crowdsourcing, swarm intelligence, etc.)
[6] Functional aspects, trustworthiness, and societal concerns
[7] AI life cycle components include acquire/process/apply.
These characteristics are identified in red in the use case.
Clarification and showcases of autonomous network usage
Short Description (up to 150 words)
Multiple scenarios of autonomous network enabled by AI is addressed for improving operational efficiency, customer experience and service innovation, including wireless network performance improvement, optical network failure prediction, data center energy saving etc.
Complete Description
The leading reason to adopt AI-assisted network automation is to reduce the cost – almost 80% operators placed this in their top three drivers, followed by:
improvement to customers’ network quality of experience
efficient planning and management of dense networks
part of an end-to-end automation strategy spanning the network and IT operations While OPEX reduction is the most important cost-related driver, others include better alignment of network costs to the revenue that is generated; and the ability to defer some capital expenditure (CAPEX) by using existing assets more efficiently.
Obviously, the autonomous self-driving network needs to move from an O&M approach that is focused on network elements, to one based on usage scenarios. This means that process changes relate directly to a particular result, defined by the operator, and with a business value. Progress will be accelerated if a core set of scenarios is defined, which will be of value to all operators. Development of the related autonomous self driving network solutions can then be prioritized accordingly.
The criteria for the selection of scenarios as follows:
Extent of digitalization: Reflects the technical readiness of the scenarios. Digitalization is the foundation of automation, and the extent to which it is supported determines the extent to which automation can be achieved immediately;
TCO contribution: Reflects OPEX savings and the improvement to CAPEX efficiency in the given scenario;
O&M life cycle: Reflects the ability to build differentiation in each phase of the life cycle in order to achieve full autonomous driving across many scenarios. The O&M life cycle spans planning, deployment, maintenance, optimization and provisioning of the network and scenarios have been identified for each one. Based on those three criteria, we selected six typical key scenarios for the purpose of illustration and clarification.
Scenario 1: Base Station Deployment
Definition and Description of Scenario The base station deployment scenario refers to the entire process after site survey, including network planning and design, site design, configuration data preparation, site installation, site commissioning and site acceptance.
Automation Classification
Level 1: The O&M tool helps some elements of the process to be automated, but configuration and site acceptance have to be done manually.
Level 2: Some hardware can be detected and configured automatically, and configuration data is simplified based on rules.
Level 3: E2E automation: radio parameter self-planning, hardware self-detection and self-configuration, self-acceptance without dialing test.
Initial outcomes: Upon the usage of AI, some initial results are achieved as follows: -Site Deployment Time Shortened by 30% -Feature Deployment Time Shortened by 60% -Performance Converging Shortened by 85%
Scenario 2: Network Performance Monitoring
Definition and Description of Scenario The mobile network has entered the stage of very precise planning sites and resources: on the one hand, to identify and forecast high traffic areas, and allocate resources precisely to support business goals; on the other hand, to identify and forecast high-frequency temporary traffic, scheduling resources to meet business objectives.
Automation Classification Level 1: Network quality is consistent, and network anomalies can be discovered by tools; Level 2: 3D presentation of network quality and anomalies, and network planning is self-generated; Level 3: E2E closed-loop monitoring and planning: predicting network development according to historical network information, finding value areas and hidden problems, recommending the best network planning and estimating the gain automatically.
Scenario 3: Fault Analysis and Handling
Definition and Description of Scenario The security and reliability is the most important mission of the network, so quick alarm detection and quick fault healing are important. The fault analysis and handling scenario comprises several steps, including alarm monitoring, root cause analysis, and fault remediation. Monitoring: Real-time monitoring of network alarm, performance, configuration, user experience, and other information. Analysis: By analyzing the correlation between alarms and other dimensions data, root cause of fault and fault repairing can be achieved quickly. Healing: Repair fault remotely or by site visiting based on the repairing suggestions.
Automation Classification Level 1: Some tools are used to simplify alarm processing, but thresholds and alarm correlation rules are set manually based on expert experience. Level 2: Automatic alarm correlation and root cause analysis. Level 3: Closed-loop of alarms analysis and handling process: Based on the intelligent correlation analysis of multi-dimensional data, accurate location of alarm root cause, precise fault ticket dispatching, and fault self-healing could be reached successfully. Level 4: Proactive troubleshooting: Based on the trend analysis of alarms, performance, and network data, alarms and faults could be predicted and rectified in advance.
Initial outcomes: Upon the usage of AI, some initial results are achieved as follows:
-Reduction of alarms: 90%
Scenario 4: Network Performance Improvement
Definition and Description of Scenario Wireless networks are geographically very distributed, and activity varies significantly in different places and at different times of day. This makes the network very dynamic and complex. That complexity is further increased by the diversity of services and of terminal performance, and by the mobility of users. If the network cannot achieve the benchmark KPIs or SLAs (service level agreements), or enable good user experience, it must be adjusted to meet or exceed those requirements.
This is the function of network performance improvement or optimization.
The complete process of network performance improvement or optimization includes several stages:
network monitoring and evaluation
root cause analysis of performance problems
optimization analysis and optimization decision-making
optimization implementation
post- evaluation and verification
Automation Classification Level 2: Drive test evaluation is not required for coverage optimization. Adjustment suggestions are provided automatically. Level 3: Closed-loop of network performance improvement:
Automatic identification of network coverage and quality problems, automatic configuration of performance parameters, and automatic evaluation.
Level 4: Dynamic adjustment is implemented based on the scenario awareness and prediction to achieve the optimal network performance. Network prediction capability is available: scenario change trends could be perceived, and network configuration could adjusted real-time to achieve optimal performance.
Initial outcomes: Upon the usage of AI, some initial results are achieved as follows:
-Capacity increase: 30%,
-Delivery duration: 2 weeks, non-manual
Scenario 5: Site Power Saving
Definition and Description of Scenario T Site power consumption cost accounts for more than 20% of network OPEX. Although network traffic declines greatly during idle hours, equipment continues to operate, and power consumption does not dynamically adjust to the traffic level, resulting in waste. It is necessary to build the "Zero Bit, Zero Watt" capability.
Automation Classification Level 2: Tool aided execution; Level 3: Power-saving closed-loop: Based on the analysis of traffic trends, self-adaptive generation of power-saving strategies, effect and closed-loop KPI feedback; Level 4: Real-time adjustment of power-saving strategies based on traffic prediction. Through integration with third-party space-time platforms, the operator can also add predictive perception of traffic changes, smooth out the user experience, and maximize power-saving.
Initial outcomes: Upon the usage of AI, some initial results are achieved as follows:
-Power saving: 10~15%
Scenario 6: Wireless Broadband Service Provisioning
Definition and Description of Scenario WTTx has become a foundational service for mobile operators because of its convenient installation and low cost of single bit. Rapid launch of WTTx service, accurate evaluation after launch, and network development planning have become important supports for new business development.
Automation Classification Level 1: Blind launch;
Level 2: Automation tools to assist the launch, check the coverage and capacity of the user's location before the business hall, and experience evaluation;
Level 3: Closed-loop for business launch: Integrated with BOSS system to achieve one-step precise launch, remote account launching, CPE installation, fault self-diagnosis and complaint analysis;
Level 4: Auto-balancing of multi-service, automatic value areas identification and network planning recommendation.
Ratio of potential issues which are "of interest" for human evaluation. Ideal target is to reduce the current volume by 80%.
Improve accuracy
1
Precision
Correctly Predicted Anomalous scenarios/ Total Anomalous scenarios predicted
1
Features related to adulterants in radio spectrum
Intensities around NIR range
1
Average vehicle driving speed
Average vehicle driving speed on all the road sections in a given region
Improve the road utilization efficiency
1
Coverage
Ratio of EMR QC requirements done in the solution/all issued EMR QC requirements in China. Ideal target is 100%.
Improve accuracy
1
Accuracy
The number of correctly recognized users’ intent over total number of users. Currently, accuracy reaches 95%.
Improve accuracy of recognizing users’ intent
1
Classifier Accuracy
Without straightening and pre-processing, the average classification accuracy obtained was 68.5%. However, with preprocessing, the classification accuracy improved to 86.7%. These results are very likely to improve with more annotated training data for classification.
1
Closeness to Golden Batch
How close a process is to the best possible batch
Helps in isolation of bad batches from good batches by identifying combination of process variable trajectories that lead to good or bad batch operation.
1
Model Accuracy
Accuracy of the prediction model
The extent to which the setpoints have correctly predicted
1
Ratio of ML discovered failure rate to nominal failure rate
What combination of manufacturing processes/decisions leads to higher failure rates compared to nominal failure rate
Actionable intelligence to improve the manufacturing process of HV circuit breakers
1
Number of labors
reduced % of labors
improvement of productivity
1
Customer Satisfaction
The ratio of customer satisfaction when using this system for requests. The expectation is 100%
Increasing its ratio as high as possible
1
MIoU (Mean Intersection over Union)
The intersection of prediction area and actual area divided by the union of the predicted area and the actual area. Ideal target is 100%.
Improve accuracy
1
Classification Ratio
Real to Pseudo wrong classification
Establishes the quality of identification
1
Ease of use
Simplicity and efficiency during initial learning. Teaching process should be easy.
1
Zone of Influence/ Thermal Correlation Index
Extent of influence of ACUs on data center racks.
Helps in improved control.
1
Invisible Loss Time
Indicates the lost time of the asset in being idle or off or unplanned downtime
Asset Utilization Reports indicate the effectively utilized time there indicating the lost time and their causes
1
Prediction Accuracy
To what extent has the model been able to predict correctly
Provided ability as to % of times the quality complied
1
Algorithm accuracy
Output when compared to the human expert analysis of the same data
See Reference
1
Generation of Activities (land use information and time of travel)
Purpose of activities is assigned based on land use information and time of travel. Census data and national/ local travel surveys will provide validation for the process
Phrase 1
1
accuracy
The accuracy of infraction and incident detection from traffic pictures/videos
To increase the accuracy of traffic monitoring and inspection
2
Split
Proportion of the potential issues which are "more likely to be a valid issue" for our end users.
Average vehicle waiting time at all the intersections in a given region
Improve the road utilization efficiency
2
Resolution
The number of answers solved over total number of questions asked
Improve the resolution of questions from users
2
Annotation Completeness
35.9 chromosomes segmented out after crowd annotation, for 50 images having 46 chromosomes
2
% Reduction in Calibration Time
The amount of time saved from manually setting the calibration
2
Number of complaints
reduced % of labor's complaint
improvement of productivity
2
Accuracy
Among all the predicted customer sentiment classification, the ratio of accurate prediction, current value is 76.4%
Increasing to 90%
2
FAR (false acceptance rate)
Negative samples are identified as positive samples / Total number of negative samples.The low FAR, the more smartphone will get correct scenes and objects
Improve accuracy
2
Training efficiency
Amount of necessary data for training might lead to practical obstacles in application.
2
Overall drilling time
The time spent on one drilling job inclusive of the all downtimes
Real Time visibility into operations gives the operations early warnings to take actions immediately.
2
Generation of agents (travel times, speed on link)
Agents generated will build up in the network creating realistic conditionsw of congestion. Speed on links.
Phrase 2
2
split
Proportion of images requiring human inspection. The less the split, the higher the efficiency.
To minimize the human effort in inspection
3
Satisfaction
The number of users who are satisfied with customer service over total number of users
Improve user experience
3
Lead time
time from order to shipment
improvement of productivity
3
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 90%
Increasing to 90%
3
Initial success rate
After initial training, the success rate needs to be acceptable such that the system can be put in the production line.
3
Opeartion of service (number of users for the service)
Optimisation of route and operation time in the day. Validation provided using data collected by Mobility service operators during the operation of service
Phrase 3
3
resource utilization ratio
Achievable resource utilization ratio in the hardware infrastructure ( the higher the utilization ratio, the lower amount the required resource)
To reduce the infrastructure investment and overall solution cost
4
Accuracy
Among all the predicted customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
4
Speed of improvement
Higher convergence speed of the reinforcement algorithm is making the solution more attractive.
5
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
5
Operational efficiency
Cycle time is the primary measure in manufacturing industry.
6
Success rate
Very high success rate is required for the solution to be accepted.
Editor's comments and enhancements are shown in green. [✓ Reviewed]
The quality of use case submissions will be evaluated for inclusion in the Working Group's Technical Report based on the application area, relevant AI technologies, credible reference sources (see References section), and the following characteristics:
[1] Data Focus & Learning: Use cases for AI system which utilizes Machine Learning, and those that use a fixed a priori knowledge base.
[2] Level of Autonomy: Use cases demonstrating several degrees (dependent, autonomous, human/critic in the loop, etc.) of AI system autonomy.
[3] Verifiability & Transparency: Use cases demonstrating several types and levels of verifiability and transparency, including approaches for explainable AI, accountability, etc.
[4] Impact: Use cases demonstrating the impact of AI systems to society, environment, etc.
[5] Architecture: Use cases demonstrating several architectural paradigms for AI systems (e.g., cloud, distributed AI, crowdsourcing, swarm intelligence, etc.)
[6] Functional aspects, trustworthiness, and societal concerns
[7] AI life cycle components include acquire/process/apply.
These characteristics are identified in red in the use case.
No.
62
ID:
Use Case Name:
A judging support system for gymnastics using 3D sensing
To support judgement of difficult element by high-level and high-speed.
Short Description (up to 150 words)
We have been developing a judging support system for artistic gymnastics to enhance accuracy and fairness in judging. We developed a skeleton recognition technique using the learned model that we trained using a large amount of depth images of gymnastics created from CG in advance. With this technology, it is possible to recognize a human 3D skeleton from depth image.
Complete Description
In gymnastics, wrong scoring is a problem, when it is difficult to judge by high-level and high-speed. Therefore, 3D sensing technology is required to reduce burden of referee by recognizing skeleton of gymnast. We developed a technique to recognize heatmaps of body parts using the learned model that we trained using a large amount of depth images of gymnastics created from CG in advance. We calculate 3D skeleton position using heatmaps of body parts. With this technology, it is possible to recognize a human 3D skeleton from depth image.
Ratio of potential issues which are "of interest" for human evaluation. Ideal target is to reduce the current volume by 80%.
Improve accuracy
1
Precision
Correctly Predicted Anomalous scenarios/ Total Anomalous scenarios predicted
1
Features related to adulterants in radio spectrum
Intensities around NIR range
1
Average vehicle driving speed
Average vehicle driving speed on all the road sections in a given region
Improve the road utilization efficiency
1
Coverage
Ratio of EMR QC requirements done in the solution/all issued EMR QC requirements in China. Ideal target is 100%.
Improve accuracy
1
Accuracy
The number of correctly recognized users’ intent over total number of users. Currently, accuracy reaches 95%.
Improve accuracy of recognizing users’ intent
1
Classifier Accuracy
Without straightening and pre-processing, the average classification accuracy obtained was 68.5%. However, with preprocessing, the classification accuracy improved to 86.7%. These results are very likely to improve with more annotated training data for classification.
1
Closeness to Golden Batch
How close a process is to the best possible batch
Helps in isolation of bad batches from good batches by identifying combination of process variable trajectories that lead to good or bad batch operation.
1
Model Accuracy
Accuracy of the prediction model
The extent to which the setpoints have correctly predicted
1
Ratio of ML discovered failure rate to nominal failure rate
What combination of manufacturing processes/decisions leads to higher failure rates compared to nominal failure rate
Actionable intelligence to improve the manufacturing process of HV circuit breakers
1
Number of labors
reduced % of labors
improvement of productivity
1
Customer Satisfaction
The ratio of customer satisfaction when using this system for requests. The expectation is 100%
Increasing its ratio as high as possible
1
MIoU (Mean Intersection over Union)
The intersection of prediction area and actual area divided by the union of the predicted area and the actual area. Ideal target is 100%.
Improve accuracy
1
Classification Ratio
Real to Pseudo wrong classification
Establishes the quality of identification
1
Ease of use
Simplicity and efficiency during initial learning. Teaching process should be easy.
1
Zone of Influence/ Thermal Correlation Index
Extent of influence of ACUs on data center racks.
Helps in improved control.
1
Invisible Loss Time
Indicates the lost time of the asset in being idle or off or unplanned downtime
Asset Utilization Reports indicate the effectively utilized time there indicating the lost time and their causes
1
Prediction Accuracy
To what extent has the model been able to predict correctly
Provided ability as to % of times the quality complied
1
Algorithm accuracy
Output when compared to the human expert analysis of the same data
See Reference
1
Generation of Activities (land use information and time of travel)
Purpose of activities is assigned based on land use information and time of travel. Census data and national/ local travel surveys will provide validation for the process
Phrase 1
1
accuracy
The accuracy of infraction and incident detection from traffic pictures/videos
To increase the accuracy of traffic monitoring and inspection
2
Split
Proportion of the potential issues which are "more likely to be a valid issue" for our end users.
Average vehicle waiting time at all the intersections in a given region
Improve the road utilization efficiency
2
Resolution
The number of answers solved over total number of questions asked
Improve the resolution of questions from users
2
Annotation Completeness
35.9 chromosomes segmented out after crowd annotation, for 50 images having 46 chromosomes
2
% Reduction in Calibration Time
The amount of time saved from manually setting the calibration
2
Number of complaints
reduced % of labor's complaint
improvement of productivity
2
Accuracy
Among all the predicted customer sentiment classification, the ratio of accurate prediction, current value is 76.4%
Increasing to 90%
2
FAR (false acceptance rate)
Negative samples are identified as positive samples / Total number of negative samples.The low FAR, the more smartphone will get correct scenes and objects
Improve accuracy
2
Training efficiency
Amount of necessary data for training might lead to practical obstacles in application.
2
Overall drilling time
The time spent on one drilling job inclusive of the all downtimes
Real Time visibility into operations gives the operations early warnings to take actions immediately.
2
Generation of agents (travel times, speed on link)
Agents generated will build up in the network creating realistic conditionsw of congestion. Speed on links.
Phrase 2
2
split
Proportion of images requiring human inspection. The less the split, the higher the efficiency.
To minimize the human effort in inspection
3
Satisfaction
The number of users who are satisfied with customer service over total number of users
Improve user experience
3
Lead time
time from order to shipment
improvement of productivity
3
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 90%
Increasing to 90%
3
Initial success rate
After initial training, the success rate needs to be acceptable such that the system can be put in the production line.
3
Opeartion of service (number of users for the service)
Optimisation of route and operation time in the day. Validation provided using data collected by Mobility service operators during the operation of service
Phrase 3
3
resource utilization ratio
Achievable resource utilization ratio in the hardware infrastructure ( the higher the utilization ratio, the lower amount the required resource)
To reduce the infrastructure investment and overall solution cost
4
Accuracy
Among all the predicted customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
4
Speed of improvement
Higher convergence speed of the reinforcement algorithm is making the solution more attractive.
5
Recall
Among all the customer sentiment intensity, the ratio of accurate prediction, current overall value is 85%
Increasing to 90%
5
Operational efficiency
Cycle time is the primary measure in manufacturing industry.
6
Success rate
Very high success rate is required for the solution to be accepted.
Train multiple models (deep learning, Bayesian network, Time series analysis) for recognizing traffic flow volume and abnormal values in the input data
1
Training
Train a model (deep neural network) with training samples
1
Training
Based on millions of labeled streaming data, train a model using diversified algorithms, such as a deep learning neural network or a traditional machine learning algorithm
1
Data Augmentation
Using reverse translation and noise processing to increase the size and diversity of data.
Increase the performance of model training.
1
Training
Train a model (e.g. neural network) with training samples
1
Training
Train a model with training data set
Evaluation
2
Optimization
Based on the data processed by the trained models, optimize the period length, split, and key phase offsets among multiple intersections for traffic signal timing plans
Completion of missing values or abnormal values processings
2
Evaluation
Evaluate whether the trained model can be deployed
2
Evaluation
Evaluate the performance of the model on online dialogue data
Each requirement must be satisfied or exceeded to reach the condition of 'success' (e.g. the accuracy should be more than 95%)
2
Model Training
Based on the large training data, with deep learning method, to develop model for sentiment classification (7 categories) or sentiment intensity (3 categories).
2
Evaluation
Evaluate whether the model is properly trained for the detection
Meeting KPI requirements (e.g. accuracy, split) of the particular case
2
Evaluation
Evaluate whether the trained model can be deployed cg data
Training/Retraining
Execution
3
Evaluation
Pre-evaluate the execution effects of the optimized traffic signal timing plans, which include the period lengths, splits, and key phase offsets among multiple intersections
Input prediction of traffic flow situation in the next period
The pre-evaluated execution effects of the optimized traffic signal timing plan is superior to the current one
3
Execution
Detect defects (regions including defects) using the trained model
The trained model has been evaluated as deployable
3
Execution
Apply the trained model to predict user’s intent
3
Evaluation
Evaluate data performance on open dataset and specific data.
3
Execution
Deploy the model for infraction detection and traffic analysis
The model has been evaluated as properly trained.
3
Execution
Recognize real data gained 3D laser sensor
Evaluation
Retraining
4
Execution
Execute the optimized traffic signal timing plan
The pre-evaluated execution effects of the optimized traffic signal timing plan is superior to the current one
4
Retraining
Retrain a model with training samples
4
Retraining
Take a training sample from online dialogue to retrain the model and compare it with the old one by AB test
The requirement is that the new model must be better than the old one
4
Execution
Apply the trained model on real-time AI customer service.
The trained model has been evaluated as deployable
Transform video data into structured data by deep learning
1
Training
Train a model (deep neural network) with training samples
Sample raw dataset is ready
1
Raw data stored in the database
Data extraction
Database engineer
Extract related data from the database to generate the raw dataset
1
Complete data augmentation
Design model for training
AI algorithm engineers
Using CNN for sentiment classification and intensity.
2
Completion of Step 1
Data clustering
AI provider
Recognize abnormal value patterns and label them in internet data, induction coil data, and structures video data by data clustering
2
Evaluation
Evaluate whether the trained model can be deployed
Completion of training/retraining
2
Completion of Step 1
Generating training samples
Data labeling team
Label the raw dataset of step one with 300 categories
2
Complete model designing
Transfer learning
AI algorithm engineers
Multi-task learning with different data in same domain.
3
Completion of Step 2?
Processing of missing value and abnormal value
AI provider
Recognize abnormal value and process them, and fill missing values by data clustering, time series analysis and Bayesian network
3
Execution
Detect defects (regions including defects) using the trained model
Completion of deployment in EMR system
The trained model has been evaluated as deployable
3
Completion of Step 2
Pre-process
AI engineer
Segment the sentence into words and convert those words into vectors
4
Completion of Step 3
Data fusion
AI provider
Compute traffic status parameters such as traffic volume, vehicle driving speed, etc. by fusing internet data, induction coil data and structured video data
4
Retraining
Retrain a model with training samples
Certain period of time has passed since the last training/retrainig
Working Group on Research and Innovation of the Plattform Industrie 4.0. Aspects of the Research Roadmap in Application Scenarios, Working Paper, German Federal Ministry for Economic Affairs and Energy, 2016.
ZHANG MAOLEI;WEI LIXIA;CHEN XIAOMING;LI JIN.?”Crossing traffic jam judging and control method and system based on sensing detectors ”.CN201310395431.2013
Sharma, Monika & Saha, Oindrila & Sriraman, Anand & Hebbalaguppe, Ramya & Vig, Lovekesh & Karande, Shirish. (2017). Crowdsourcing for Chromosome Segmentation and Deep Classification. 786-793. 10.1109/CVPRW.2017.109.
1
Working Group on Research and Innovation of the Plattform Industrie 4.0. Aspects of the Research Roadmap in Application Scenarios, Working Paper, German Federal Ministry for Economic Affairs and Energy, 2016.
Working Group on Research and Innovation of the Plattform Industrie 4.0. Aspects of the Research Roadmap in Application Scenarios, Working Paper, German Federal Ministry for Economic Affairs and Energy, 2016.
Kumar, S., K., Jamkhandi, A., G., and Gugaliya, J., K., Achieving Manufacturing Excellence through Data Driven Decisions, IEEE International Conference on Industrial Technology, Melbourne Australia PP 1267-1273
Jeffy, F., J., Gugaliya, J., K., and Kariwala, V. Application of Multi-Way Principal Component Analysis on Batch Data, 2018 UKACC 12th International Conference on Control
B. Kuhlenkötter, X. Zhang, C. Krewet, Quality Control in Automated Manufacturing Processes – Combined Features for Image Processing Acta Polytechnica Vol. 46 No. 5/2006.
Fan Dai, Arne Wahrburg, Björn Matthias, Hao Ding: Robot Assembly Skills Based on Compliant Motion Proceedings of 47th International Symposium on Robotics (ISR 2016), At Munich, Germany
Causality-based Thermal Prediction for Data Center. 2018 IEEE 23rd International Conference on Emerging Technologies and Factory Automation (ETFA). Turin, Italy. 4-7 Sept. 2018.
Franco P, Johnston R, McCormick (2019) Demand Responsive transport: generation of activity patterns from mobile phone network data to support the operation of flexible mobility services. - Special issue of Transportation Research Part A (TRA) on developments in Mobility as a Service (MaaS) and intelligent mobility (forthcoming)
Sakti Saurav, Pankaj Malhotra, Vishnu TV, Narendhar Gugulothu, Lovekesh Vig, Puneet Agarwal, Gautam Shroff, Online anomaly detection with concept drift adaptation using recurrent neural networks, CoDS-COMAD '18, Proceedings of the ACM India Joint International Conference on Data Science and Management of Data, Goa, India — January 11 - 13, 2018
2
Working Group on Research and Innovation of the Plattfom Industrie 4.0 and Alliance Industrie du Futur: Plattform Industrie 4.0 & Alliance Industrie du Futur : Common List of Scenarios. 2018.
Working Group on Research and Innovation of the Plattfom Industrie 4.0 and Alliance Industrie du Futur: Plattform Industrie 4.0 & Alliance Industrie du Futur : Common List of Scenarios. 2018.
Working Group on Research and Innovation of the Plattfom Industrie 4.0 and Alliance Industrie du Futur: Plattform Industrie 4.0 & Alliance Industrie du Futur : Common List of Scenarios. 2018.
Te Tang, Hsien-Chun Lin, Masayoshi Tomizuka, A learning-based framework for robot peg-hole-insertion, Proceedings of the ASME 2015 Dynamic Systems and Control Conference, October 28-30, 2015, Columbus, Ohio, USA
Franco P, Johnston R, McCormick E (2018) Role of Intelligent Transport Systems applications in the uptake of mobility on demand services, United Nation “Transport and Communications Bulletin for Asia and the Pacific, 2018, No. 88 - Intelligent Transport Systems”.
Louise Radnofsky. The Robots Are Coming (to Judge Gymnastics). The international gymnastics federation may add new technology in time for the 2020 Olympics—and pave the way for artificial intelligence in judged sports. The Wall Street Journal, Aug. 23, 2019.
Communication Promoters Group of the Industry-Science Research Alliance and German National Academy of Science and Engineering. Recommendations for implementing the strategic initiative INDUSTRIE 4.0, Final report of the Industrie 4.0 Working Group, April 2013.
Communication Promoters Group of the Industry-Science Research Alliance and German National Academy of Science and Engineering. Recommendations for implementing the strategic initiative INDUSTRIE 4.0, Final report of the Industrie 4.0 Working Group,, April 2013
Communication Promoters Group of the Industry-Science Research Alliance and German National Academy of Science and Engineering. Recommendations for implementing the strategic initiative INDUSTRIE 4.0, Final report of the Industrie 4.0 Working Group, April 2013.
Fares J. Abu-Dakka, Bojan Nemec, Aljaž Kramberger, Anders Glent Buch, Norbert Krüger and Aleš Ude, Solving peg-in-hole tasks by human demonstration and exception strategies, Industrial Robot: An International Journal 41/6 (2014) 575–584
Published
Cited to support the detailed description
Jožef Stefan Institute, Dept. of Automatics, Biocybernetics, and Robotics, Slovania, Maersk Mc-Kinney Moller Institute, University of Southern Denmark
Franco P, McCormick E, Johnston R (2018) Multimodal activity Modelling for supporting mobility service operations, ITS World Congress Copenhagen, 17-21 September 2018
4
Bo-hu LI, Bao-cun HOU, Wen-tao YU, Xiao-bing LU, Chun-wei YANG. Applications of artificial intelligence in intelligent manufacturing: a review. Frontiers of Information Technology & Electronic Engineering. 2017
4
patent
HUA XIANSHENG” Assessment method and device of traffic condition”. CN201610645412.2016
Christoph Legat and Birgit Vogel-Heuser. A configurable partial-order planning approach for field level operation strategies of PLC-based industry 4.0 automated manufacturing systems. Engineering Applications of Artificial Intelligence 66:128-144, DOI: 10.1016/j.engappai.2017.06.014,02017.
4
Birgit Vogel-Heuser, Jay Lee, and Paolo Leitao. Agents enabling cyber-physical production systems. at – automatisierungstechnik 63(10). DOI: 10.1515/auto-2014-1153, 2015.
4
Publication
?Mel Vecerik, Todd Hester, Jonathan Scholz, Fumin Wang, Olivier Pietquin, Bilal Piot, Nicolas Heess, Thomas Rothörl, Thomas Lampe, Martin Riedmiller, Leveraging Demonstrations for Deep Reinforcement, Learning on Robotics Problems with Sparse Rewards, arXiv:1707.08817v2 [cs.AI] 8 Oct 2018
Franco P, McCormick E, Van Leeuwen K, Ryan Johnston, Gregor Engelmann (2017) Multi-Modal Activity-Based Models to support Flexible Demand Mobility Services. ITS World Congress 2017, Montreal 29 October- 2 November 2017. Awarded Best Paper
5
Lee, Jay, Hung-An Kao, and Shanhu Yang. "Service innovation and smart analytics for industry 4.0 and big data environment." Procedia Cirp 16 (2014): 3-8.
5
patent
HUA XIANSHENG,REN PEIRAN,SHEN CHEN,CHU WENQING,LIU YAO.” Road intersection traffic flow control method and device”. CN201610644132.2016
Birgit Vogel-Heuser, Jay Lee, and Paolo Leitao. Agents enabling cyber-physical production systems. at – automatisierungstechnik 63(10). DOI: 10.1515/auto-2014-1153, 2015.
5
Weyer, Stephan, et al. "Towards Industry 4.0-Standardization as the crucial challenge for highly modular, multi-vendor production systems." Ifac-Papersonline 48.3 (2015): 579-584.
5
Publication
Mel Vecerik, Oleg Sushkov, David Barker, Thomas Roth¨orl, Todd Hester, Jon Scholz, A Practical Approach to Insertion with Variable Socket Position Using Deep Reinforcement Learning, arXiv:1810.01531v2 [cs.RO] 8 Oct 2018
Franco P, McCormick E, Van Leeuwen K (2017) Framework for modelling MaaS using ABM and real-time data from ride-sharing services. 12ve ITS Europe Congress 2017, Strasbourg, 19-22 June 2017.
6
paper
Liang Yu,Jingqiang Yu,Maolei Zhang?Xin Zhang?Yuehu Liu.”Large Scale Traffic Signal Network Optimization-a Paradigm Shift Driven by Big Data”. ICDE2019
Alibaba Cloud Computing Hangzhou?China
6
Patent
Medical record text structure analysis algorithm and device based on pseudo corpus generation
Jens Otto and Oliver Niggemann. Automatic Parameterization of Automation. Software for Plug-and-Produce. AAAI Workshop on Algorithm Configuration, 2015
6
Lasi, Heiner, et al. "Industry 4.0." Business & information systems engineering 6.4 (2014): 239-242
7
M. Papageorgiou, C. Diakaki, V. Dinopoulou, A. Kotsialos, and Y.Wang, “Review of road traffic control strategies,” Proceedings of the IEEE, vol. 91, no. 12, pp. 2043–2067, 2003.
7
Patent
Algorithm and device for improving accuracy of medical record quality assurance system by using doctor behavior log
Christoph Legat, Christian Seitz, Steffen Lamparter und Stefan Feldmann. Semantics to the Shop Floor: Towards Ontology Modularization and Reuse in the Automation Domain. IFAC Processings, Vol. 47, Issue 3, pp. 3444 – 3449, Doi:10.3182/20140824-6-ZA-1003.02512, 2014.
7
Monostori, László. "Cyber-physical production systems: Roots, expectations and R&D challenges." Procedia Cirp 17 (2014): 9-13.
8
paper
P. Lowrie, “Scats, sydney co-ordinated adaptive traffic system: A traffic responsive method of controlling urban traffic,” 1990.
8
Patent
Medical record text structure analysis algorithm and device based on context-free grammar parsing technology
Martin Ringsquandl, Steffen Lamparter, Sebastian Brandt, Thomas Hubauer, and Raffaello Lepratti. Semantic-Guided Feature Selection For Industrial Automation Systems. International Semantic Web Conference. Springer, Cham, 2015.
9
paper
F. Corman, A. D’Ariano, D. Pacciarelli, and M. Pranzo, “Evaluatio of green wave policy in real-time railway traffic management,” Transportation Research Part C: Emerging Technologies, vol. 17, no. 6, pp. 607–616, 2009.
9
Patent
Algorithm and device for structural analysis of medical records combined with visual features
L. Singh, S. Tripathi, and H. Arora, “Time optimization for traffic signal control using genetic algorithm,” International Journal of Recent Trends in Engineering, vol. 2, no. 2, p. 4, 2009.
10
Patent
Method and device for Chinese medical record named entity recognition by using Iterated Dilated CNN with condition random field model based on Chinese character structure
Method and device to detect similar electronic medical records
In application
Peer-reviewed scientific/technical publications on AI applications (e.g. [1]).
Patent documents describing AI solutions (e.g. [2], [3]).
Technical reports or presentations by renowned AI experts (e.g. [4])
High quality company whitepapers and presentations
Publicly accessible sources with sufficient detail
This list is not exhaustive. Other credible sources may be acceptable as well.
Examples of credible sources:
[1] B. Du Boulay. "Artificial Intelligence as an Effective Classroom Assistant". IEEE Intelligent Systems, V 31, p.76-81. 2016.
[2] S. Hong. "Artificial intelligence audio apparatus and operation method thereof". N US 9,948,764, Available at: https://patents.google.com/patent/US20150120618A1/en. 2018.
[3] M.R. Sumner, B.J. Newendorp and R.M. Orr. "Structured dictation using intelligent automated assistants". N US 9,865,280, 2018.
[4] J. Hendler, S. Ellis, K. McGuire, N. Negedley, A. Weinstock, M. Klawonn and D. Burns. "WATSON@RPI, Technical Project Review".
URL: https://www.slideshare.net/jahendler/watson-summer-review82013final. 2013