Intelligent User Interfaces

Question 1

Intelligent User Interfaces: In the UI

Answer 1

• In the UI: Intelligent processing is found in the user interface(s) of the system, and its purpose is to enable effective, natural, or otherwise appropriate interactions with users of the system
  
  • The user interfaces are automatically adapted to the inferred capabilities, needs of the user or context of the interaction (e.g. Gmail Smart Compose)
  • Multimodal and sensing-based systems that aim to enable more natural, human-like forms of input and output (e.g. Speech, Body language)

Question 2

Intelligent User Interfaces: Behind the UI

Answer 2

• Behind the UI: Intelligent processing is found in the backend of the system, and its purpose is to serve some beneficial functions, such as performing actions on behalf of the user or modelling the user for personalisation
  
  • Recommender systems that provide us with personalised options (e.g. Netflix, Spotify)
  • Agents that perform complex or repetitive tasks with some guidance from the user
• Situated assistance systems that monitor and support user activities (e.g. Health andactivity monitoring)

Question 3

Intelligent User Interfaces

Answer 3

• IUIs employ a variety of AI technologies in order to assist users achieve their goals and
  requirements
• IUIs assist users by …
  
  • using AI to optimise the interaction by adapting to the user and their context
  • using AI to enable users to interact with the interface similar to how humans interact, making it more intuitive and user-friendly
  • using AI to intelligently provide personalised and relevant suggestions, recommendations, and responses based on their individual preferences and needs

Question 4

Example: Precise Touch

Answer 4

• How accurate is touch input our smartphones?
  
  • Touch is inherently inaccurate on smartphone screens
• Adapt keyboard to typist
• Probilistic key model
• Language model influence (e.g. bigram for english)
• Context adaptations

-Using AI to improve touch input itself

Question 5

Combining AI to optimise interaction

Answer 5

• Combination of AI technologies employed for precise touch

1. User modelling to adapting the keyboard based on the typing behaviour of the user
2. Machine learning and data from other sensors to determine the context of the user
3. Computer vision and deep learning to increase the precision of the touch input
4. Natural language processing, where a language model is to predict the likelihood of what the user will type based on the language and the key that they are likely to hit next

• IUIs are often designed to be invisible to the user
  
  • i.e. integrated seamlessly into the user experience, allowing users to interact with the
    technology in a natural and intuitive way without being aware of the underlying technology
  • Combination of using AI in and behind the UI to create such experiences

Question 6

Smart Compose and Replies

Answer 6

Question 7

Natural Language Processing

Answer 7

• Natural Language Processing (NLP) is a field of artificial intelligence (AI) that focuses on the ability of computers to understand and interpret human language, and to read and interpret text to extract information
• Have you used NLP today?
  
  • Every time you search for something and suggestions appear (autocomplete)
  • Every time you make a mistake while typing (autocorrect)
  • Every time you use a virtual assistant (speech-to-text, text understanding)
• Key enabling technology for many IUIs
  
  • Examples: virtual assistants (Siri, Alexa), chatbots, search engines

Question 8

Natural Language Processing - Timeline

Answer 8

Question 9

Natural Language Processing - Evolution

Answer 9

Question 10

NLP: Large Language Models

Answer 10

• Large language models can generate human-like text, trained on amounts of text data in order to learn the patterns and structures
  of human language
• Current large language models cannot tell fact from fiction yet, resulting in biased and incorrect results, or even false and misleading information
• Space bears, it’s easy to spot fiction when it involves space bears, but harder with a subject users may not know much about
• Can you trust an IUI built on a large language model that may provide you with false or misleading information?

Question 11

Natural Language Processing

Answer 11

• Other applications areas of natural language processing in IUI context
  • AI writing tools - Copyai (text generation), Grammarly (text editor), Quillbot (text summarisation)
  • Chatbots and virtual personal assistants - Siri, Alexa, Cortana, Messenger
  • Search engines - Google, Bing
  • Automated language translators -
    Google Translate
  • Social media - Twitter (text analysis for
    personalised content)

Question 12

Recommender Systems

Answer 12

• The goal of a recommender system is to generate meaningful recommendations to a collection of users for items or products that might interest them
• Why are recommender systems used?
  
  1. The system has more information that it can present (millions of options to display)
  2. Collecting a subset of items out of a bigger number of items (collection of songs)
  3. Ordering items from a subset - Which one should be shown first?
• How do recommender systems impact the user experience?
  
  • If it’s perfect: This is “exactly” what I wanted.

Question 13

Recommender Systems #2

Answer 13

• User modelling refers to the process of collecting and analysing data about a user’s abilities, preferences, and behavior in order to create a model of their characteristics and needs
• Collaborative filtering is a technique that uses the ratings and preferences of all users in a
  system to make recommendations
  
  • “people who liked what you liked also liked X, hence I suggest X to you”
• Content-based filtering is a technique that uses the characteristics of the items in a system
  (such as the genres, actors, and directors of movies in a streaming video service) to recommend similar items to a user.
  
  • “X is similar (based on similar measure for a domain) to what you liked before, hence I
    suggest X to you”

Question 14

What are the benefits of using IUI

Answer 14

1. Provide personalised experiences
2. Prevent information overload
3. Provide proactive assistance
4. Enables natural interaction

Question 15

Benefit: Provide personalised experiences

Answer 15

No two persons are the same and people have different habits, preferences, and working methods and environments. IUIs that take these differences into account can provide a personalised method of interaction. The interface knows the user and can use that knowledge in its communication with the user.

Question 16

Benefit: Prevent information overload

Answer 16

Finding the right information on the computer or on the internet can be like looking for a needle in a haystack. IUIs can reduce the information overload that often results from finding information in large databases or complex systems. By filtering out irrelevant information, the interface can reduce the cognitive load on the user.

Question 17

Benefit: Provide proactive assistance

Answer 17

Intelligent help systems can detect and correct user misconceptions, explain new concepts, and provide information to simplify tasks. Such systems that employ sensing technologies can proactively take action, such as calling 999 for help. IUIs with autonomous capabilities that observe users, understand and recognise their intent, and take over some of their tasks completely, allowing them to focus on other things.