Developer Console

WSA App UX: Mapping device inputs with Windows Subsystem for Android™️

Giovanni Laquidara Mar 02, 2023

Important: Amazon Appstore on Windows 11 will no longer be supported after March 5, 2025. Read the blog for more information.

As an Android developer, you may have launched mobile apps and games for multiple devices, but what about the Windows Subsystem for Android™? Bring your app to new desktop customers equipped with larger screens, keyboards, trackpads, and gaming controllers through this new experience.

In this article, we'll cover how to map these device inputs in order to unlock the full capabilities on Android apps running on Windows 11. For all of the following, we recommend you test on actual devices to ensure the best user experience

Input by Keyboard

Inputs for apps and games are determined by the UI toolkit in use. For this guide, we’ll focus on the most commonly used interfaces in Android apps. Text inputs handled by virtual keyboard like EditText work seamlessly with physical keyboards.

Include support for ‘WASD’ keys

“WASD” keys vs Arrow keys

PC games often support movement via the WASD keys on a keyboard. The following code snippet shows how to detect a WASD keypress to initiate a subroutine:

// Kotlin snippet for WASD controls

override fun onKeyUp(keyCode: Int, event: KeyEvent): Boolean {
  return when(keyCode) {
    KeyEvent.KEYCODE_W-> {
        goForward() // W pressed
        true
      }
    }
    KeyEvent.KEYCODE_A-> {
        turnLeft() // A pressed
        true
      }
    }
     KeyEvent.KEYCODE_S-> {
        if (event.isShiftPressed) {
            crouch() // Shift+S pressed
            true
        }
        else {
            goBack() // S pressed
            true
        }
      }
    }
     KeyEvent.KEYCODE_D-> {
        turnRight() // D pressed
        true
      }
    }
    else -> super.onKeyUp(keyCode, event)
  }
}

Keyboard Navigation

Keyboard navigation between UI widgets is handled by the Android Framework. If you create a custom view, make sure to make it focusable and able to handle navigation. Follow the Android documentation on focus handling for more information.

For non-editable text, consider making it selectable by adding textIsSelectable="true"; to your TextView if you are using the Android XML for UI.

<TextView
    android:id="@+id/imselectable"
    android:layout_width="wrap_content"
    android:layout_height="wrap_content"
    android:textIsSelectable="true" />

When building apps with Jetpack Compose, make sure to add a SelectionContainer function for easier controls

import androidx.compose.foundation.layout.Column
import androidx.compose.material.Text
import androidx.compose.foundation.text.selection.SelectionContainer

SelectionContainer {
    Column {
        Text("Text 1")
        Text("Text 2")
        Text("טקסט 3")
    }
}

Shortcuts

Make sure to provide keyboard shortcuts in your app, as users may prefer to use these over other inputs.

Shortcuts can be implemented using dispatchKeyShortcutEvent(), which intercepts all meta-key combinations such as Alt, Ctrl, and Shift.

Check for a specific meta-key, use these KeyEvent methods:

KeyEvent.isCtrlPressed()

KeyEvent.isShiftPressed()
KeyEvent.isAltPressed()

KeyEvent.hasModifiers()

Implementing keyboard shortcuts with Kotlin

override fun dispatchKeyShortcutEvent(event: KeyEvent): Boolean {
  return when (event.keyCode) {
    KeyEvent.KEYCODE_F -> {
        if (event.isCtrlPressed) {
            findAction() // Ctrl+F Open a Find Dialog
            true
        }
    }
    else -> {
      return super.dispatchKeyShortcutEvent(event)
    }
  }
}

Implementing keyboard shortcuts with Jetpack Compose

With Jetpack Compose you can define a modifier:

fun Modifier.interceptKey(key: Key, onKeyEvent: () -> Unit): Modifier {
    return this.onPreviewKeyEvent {
        if (it.isCtrlPressed && it.key == key && it.type == KeyUp) {
            onKeyEvent()
            true
        } else it.key == key
    }
}

...

MyWidget(
    ... ,
    modifier = Modifier
        .interceptKey(Key.F) {
            findAction() // Ctrl+F Open a Find Dialog
        }
)

Input by Mouse and trackpad

Android apps on Windows Subsystem for Android™ have the ability to incorporate mouse and trackpad inputs with an on-screen pointer. To enhance the user experience, we recommend supporting hover events in your app's UI. For example, you should visually highlight active and hover state widgets when the pointer moves. Many common widgets such as Buttons already have hover support built in.

Below is an example of an Android app running on Windows 11, demonstrating support for both active and hover states

Handling hover events in a custom view

Code for handling events:

// Kotlin
yourView.setOnHoverListener { view, event ->
    val action = event.actionMasked
    when (action) {
        ACTION_HOVER_ENTER -> {
            // Hover enter 
            return@OnHoverListener true
        }
        ACTION_HOVER_EXIT -> {
            // Hover exit 
            return@OnHoverListener false
        }
    }
    false
}

For Jetpack Compose you can handle these events by using the 'hoverable' modifier:

val interactionSource = remember { MutableInteractionSource() }
val isHovered by interactionSource.collectIsHoveredAsState()
val borderColor = if (isHovered) AmazonOrange else Color.White

Text(
    text = "I'm Hoverable", 
    modifier = Modifier
        .hoverable(interactionSource = interactionSource)
        .background(borderColor, RectangleShape)
        .padding(10.dp)
)

Mouse and trackpad users will expect to have built-in support for mouse buttons. Secondary clicks can present full menus using “contextual menus”. Use the showContextMenu to support the secondary click button in your app.

yourView.setOnContextClickListener {
  showContextMenu()
  true
}

For Jetpack Compose detection of secondary clicks, check out this code sample from our Developer Advocates on Stack Overflow and the official Android documentation to learn more on how to build this UI pattern in your app.

Game development input

For game developers, handling controller input on Windows Subsystem for Android™ is crucial for a seamless gaming experience. Refer to the official documentation to know more about how to handle controller actions.

If you developed your game with Unity you can use the Input System mapping to customize each action to the user's preferred input system: