Monitor Screen Touch Event in Android

本文介绍了一种方法,在Android应用的后台场景下捕捉屏幕触摸事件,通过定义一个极小的虚拟UI碎片来实现,该碎片放置在屏幕的一角并监听所有外部触摸事件。文章详细解释了如何创建、配置和实现这一解决方案,以及如何将其整合到Service中以保持持续监听。

http://jhshi.me/2014/11/09/monitor-screen-touch-event-in-android/index.html


In one of my projects I need to track every screen touch event in background.That is, my app needs to be "invisible" while capturing every screen touch. Hereis how I achieved this.

The idea is to define a dummy UI fragment that is really tiny (say, 1x1 pixel),and place it on one of the corners of the screen, and let it listen on all touchevents outside it. Well, literally, it's not "invisible", in fact it's inforeground all the time! But since it's so tiny so hopefully users won't feel adifference.

First, let's create this dummy view:

mWindowManager = (WindowManager) mContext.getSystemService(Context.WINDOW_SERVICE);
mDummyView = new LinearLayout(mContext);

LayoutParams params = new LayoutParams(1, LayoutParams.MATCH_PARENT);
mDummyView.setLayoutParams(params);
mDummyView.setOnTouchListener(this);

Here we set the width of the dummy view to be 1 pixel, and the height to beparent height. And we also set up a touch event listen of this dummy view, whichwe'll implement later.

Then let's add this dummy view.

LayoutParams params = new LayoutParams(
        1, /* width */
        1, /* height */
        LayoutParams.TYPE_PHONE,
        LayoutParams.FLAG_NOT_FOCUSABLE | 
        LayoutParams.FLAG_NOT_TOUCH_MODAL |
        LayoutParams.FLAG_WATCH_OUTSIDE_TOUCH,
        PixelFormat.TRANSPARENT
        );
params.gravity = Gravity.LEFT | Gravity.TOP;
mWindowManager.addView(mDummyView, params);

The key here is the FLAG_WATCH_OUTSIDE_TOUCH flag, it enables the dummy viewto capture all events on screen, whether or not the event is inside the dummyview or not.

Finally, let's handle the touch event by implementing View.OnTouchListenerlistener.

@Override
public boolean onTouch(View v, MotionEvent event) {
    Log.d(TAG, "Touch event: " + event.toString());

    // log it

    return false;
}

We need to return false since we're not really handling the event, so that theunderlying real UI elements can get those events.

A final note is that, to keep our dummy view always listening touch events, weneed to wrap all these in an Service: we create the dummy view in onCreateand add it to screen in onStartCommand. And the service should implementView.OnTouchListener to receive the touch events.

vdev virtio-input QAVF2.0User's GuideArchitectureConfigurationDeveloper Updated: October 28, 2024 Create an interface for input devices Synopsis vdev virtio-input [display display_id] [force_dev_presence enabled] [global_alpha number] [loc addr intr guest_intr] [offset x_offset,y_offset] [options [+|-]bitmask] [pipeline number] [protocol b] [sched priority] [size x_dimension,y_dimension] [window window_id] [zorder zorder] screen input_type Options display display_id Create an internal transparent window on the specified display to catch all input events from this window and forward them to the guest. This option is ignored if window is defined, because an external window is used in this case. If you don't provide display or window, by default, an internal window on the default display is created. force_dev_presence enabled Specify true if it's necessary to initialize the front end when no physical device is present. If not specified, the default value is false. Currently, this is supported only by the keyboard device. global_alpha number Specify the global alpha (SCREEN_PROPERTY_GLOBAL_ALPHA) for the internal transparent window. This option is ignored if the window option is defined, because an external window is used in this case. intr guest_intr Set the guest interrupt for this virtual device; include this option only if you also include the loc option. The guest_intr argument consists of two colon-separated components: the identifier of the name of the guest device Programmable Interrupt Controller (PIC) the input line that is asserted when the interrupting device wishes to raise an interrupt For example: gic:43 For more information on setting guest interrupts, see the section on “Configuring guests” in the QNX Hypervisor User's Guide. loc addr Set the device space address to addr. The addr argument is typically specified in hexadecimal. If you include this option, the vdev presents itself to the guest as a memory-mapped I/O (MMIO) device at the location specified by addr. Otherwise, the vdev presents itself as a PCI device. You must use this option with the intr option. offset x_offset,y_offset Adjust the x and y coordinates of all events by x_offset and y_offset, respectively. Both arguments are specified in pixels. The guest will see the events at the adjusted coordinates (x+x_offset, y+y_offset). In most cases, you don't need to use this option. It can be useful, however, if the display coordinates of the guest's window are offset from those of the hardware display. options [+|-]bitmask Apply the specified bitmask of options. The optional prefixes, + and -, add or clear the bits in the default bitmask. For instance, if you specify options +0x80, this sets bit 7 (VI_DEV_SENSITIVITY_CONTINUE) if it's not already set. Similarly, if you specify options -0x80, this unsets bit 7 if it's not already unset. If neither prefix is used (e.g., options 0x80), then options would be set to the bitmask (0x80). By default, options is set to 0x12. The rest of the bits are set according to the device. For instance, if the virtio-input vdev presents itself as a mouse (see the screen option), it sets the VI_DEV_POINTER_AS_MOUSE bit when it receives a SCREEN_EVENT_POINTER event. In most cases, you don't need to use the options option. Option Bitmask VI_DEV_EVSYN_MULTIPLE 0x0001 VI_DEV_EVSYN_ALWAYS 0x0002 VI_DEV_POINTER_AS_MOUSE 0x0004 VI_DEV_POINTER_HAS_WHEEL 0x0008 VI_DEV_0FILL (where 0 is zero) 0x0010 VI_DEV_TOUCH_ST 0x0020 VI_DEV_TOUCH_MT 0x0040 VI_DEV_SENSITIVITY_CONTINUE 0x0080 VI_DEV_ALL_OPTIONS 0x00ff pipeline number Specify the pipeline (SCREEN_PROPERTY_PIPELINE) for the internal transparent window. This option is ignored if the window option is defined, because an external window is used in this case. protocol b Change the protocol from type A to type B. The default protocol is type A. This option makes the vdev use the part of the multi-touch (MT) protocol that supports type B devices, which are those devices capable of tracking identifiable contacts. In this case, the protocol describes how to send updates for individual contacts via event slots. For more information, see https://www.kernel.org/doc/Documentation/input/multi-touch-protocol.txt. Earlier Android versions up to version 11 do not by default support type B devices. To support these devices in earlier versions, you must apply the following kernel patch: https://github.com/torvalds/linux/commit/16c10bede8b3d8594279752bf53153491f3f944f. For later Android versions, this patch is not needed. sched priority Set the priority of the pulses that the VM generates to indicate that new input is available. The default is 10. In most cases, you can omit this option. screen input_type Set the device input type. This option is required. This option tells the virtual device how to present itself to the guest driver. You can specify one of the following values: keyboard Appear as a keyboard device. mouse|pointer Appear as a pointing device with buttons. Specifying mouse tells the virtual device to issue relative coordinates; specifying pointer tells it to issue absolute ones. touch|touchscreen single-touch|multi-touch Appear as a touchscreen device. The touch and touchscreen arguments are aliases; either form tells the virtual device to appear as a touchscreen. If you specify single-touch, the virtual device captures only the first touch point of every multitouch event; if you specify multi-touch, the virtual device captures all the touch points of every multitouch event, up to a maximum of 5 points. If you specify neither of these arguments, the virtual device automatically selects the protocol according to the reported maximum number of touch points. Here's an example of how you can advertise the virtual device as a multitouch device: screen touch screen multi-touch * Capture all input events. The guest will see all supported input devices and receive the events related to them. size x_dimension,y_dimension Advertise the physical size of an absolute-coordinate device (either pointer or touchscreen) as x_dimension and y_dimension. Both arguments are specified in pixels. If you specify the window option, the vdev can automatically determine the physical size of the device and you can omit the size option. Otherwise, you must specify the size option. window window_id Read all input events from the specified external window and forward them to the guest. The window_id argument takes the SCREEN_PROPERTY_ID_STRING of the window that you want to read events from. See the Screen Developer's Guide to learn more about this string and how to retrieve it. If you omit this option, the vdev creates a full-screen input region, reads all input events from the region, and then forwards them to the guest. zorder zorder Specify the z-order (SCREEN_PROPERTY_ZORDER) for the internal transparent window. The default value for this window is 1000. This option is ignored if window is defined, because an external window is used in this case. Description (ARM and x86) The input vdev creates an interface for input devices. It supports USB keyboards, USB mice, and touch input devices. Normally, the vdev appears as a PCI device on x86. But if you specify the loc and intr options, the guest will see the vdev as a memory-mapped I/O (MMIO) device at the location specified by loc. You can specify up to three input devs in a VM configuration, where each vdev presents itself as a different device to the guest. See “Configuration examples” for more information. At least one physical input device must be connected to your host system. The vdev expects to find an input device; if it doesn't, the vdev prevents the qvm process from starting.
08-26
<think>我们正在处理一个关于配置vdev virtio-input以创建虚拟输入设备接口的问题。用户希望了解如何配置显示、屏幕类型以及MMIO或PCI设置。 首先,我们需要理解vdev virtio-input的基本概念。virtio-input是Virtio规范的一部分,用于虚拟化输入设备(如键盘、鼠标)。在QEMU/KVM环境中,我们可以通过命令行选项来配置virtio-input设备。 用户提到的配置包括: 1. 显示(display):可能指将虚拟输入设备关联到哪个显示设备(如SDL、GTK、VNC等)。 2. 屏幕类型(screen type):可能指与输入设备关联的屏幕(例如多个显示器的情况)。 3. MMIO或PCI设置:virtio设备可以配置为使用MMIO(内存映射I/O)或PCI传输方式。 由于virtio-input设备通常作为PCI设备呈现给虚拟机,因此我们主要关注PCI配置。但在某些架构(如ARM)上,可能使用MMIO。 在QEMU中,添加一个virtio-input设备的基本步骤如下: a) 指定输入设备类型(如键盘、鼠标、平板等)。 b) 将输入设备与后端(如宿主机的输入设备或虚拟输入事件)关联。 c) 如果需要,指定显示选项。 然而,virtio-input设备本身并不直接与显示绑定。输入设备是独立于显示设备的,但通常我们会将输入设备发送到某个显示设备(例如,通过指定显示设备为virtio-gpu-pci时,输入事件会传递到该GPU关联的显示)。 因此,我们可能需要创建一个virtio输入设备,并将其与一个virtio GPU设备关联(如果使用virtio GPU)。virtio GPU设备可以配置显示参数。 下面是一个示例QEMU命令行,用于创建一个带有virtio键盘和鼠标的虚拟机,并使用virtio GPU: ```bash qemu-system-x86_64 \ -device virtio-gpu-pci,id=vgpu0 \ -device virtio-keyboard-pci,id=vkbd0,display=vgpu0 \ -device virtio-mouse-pci,id=vmouse0,display=vgpu0 \ ... # 其他参数 ``` 这里,`display=vgpu0`表示将键盘和鼠标的输入事件发送到virtio-gpu-pci设备(id为vgpu0)所关联的显示。 关于屏幕类型:如果使用多个显示器,我们可以在virtio-gpu设备上配置多个heads(即输出)。例如: ```bash -device virtio-gpu-pci,id=vgpu0,max_outputs=2 ``` 然后,我们可以通过显示后端(如SDL、GTK)来管理多个屏幕。输入设备通常与主屏幕交互,但也可以根据具体配置将输入事件定向到特定屏幕。不过,virtio-input设备本身没有直接设置屏幕类型的选项,它通过关联的显示设备(如virtio-gpu)来传递事件。 关于MMIO或PCI:在QEMU中,我们通常使用`-device`来添加设备,设备模型默认为PCI(对于x86)。对于ARM等架构,我们可以使用`-device virtio-keyboard-device`(MMIO)形式。但注意,virtio-input设备在PCI和MMIO下的设备名称不同: - PCI: `virtio-keyboard-pci`, `virtio-mouse-pci`, `virtio-tablet-pci` - MMIO: `virtio-keyboard-device`, `virtio-mouse-device`, `virtio-tablet-device` 因此,根据架构选择适当的设备类型即可。 总结配置步骤: 1. 确定虚拟输入设备的类型(键盘、鼠标等)。 2. 创建一个virtio GPU设备(如果需要显示)并配置其显示参数(如max_outputs)。 3. 创建virtio输入设备,并通过`display`属性将其关联到virtio GPU设备。 4. 根据架构选择PCI或MMIO设备类型。 注意:如果不需要图形界面(即不需要关联显示),则可以不指定`display`属性,此时输入设备将作为普通的输入设备(例如,可以通过控制台使用)。 另外,关于显示后端的设置(如VNC、SDL、GTK等)是在QEMU的显示选项中配置的,例如: ```bash -vnc :0 # 或 -display sdl # 或 -display gtk ``` 这些显示后端将负责渲染virtio GPU的输出,并处理输入事件。 因此,完整的配置示例(使用VNC显示和PCI设备)可能如下: ```bash qemu-system-x86_64 \ -vnc :0 \ -device virtio-gpu-pci,id=vgpu0,max_outputs=1 \ -device virtio-keyboard-pci,id=vkbd0,display=vgpu0 \ -device virtio-mouse-pci,id=vmouse0,display=vgpu0 \ ... # 其他设备(如内存、CPU、磁盘等) ``` 在这个例子中,我们通过VNC显示,并且将键盘和鼠标的输入事件定向到virtio-gpu设备(vgpu0),该设备负责将输入事件传递给VNC客户端。 如果用户希望使用MMIO(例如在ARM virt机器上),则配置可能如下: ```bash qemu-system-aarch64 \ -machine virt \ -vnc :0 \ -device virtio-gpu-device,id=vgpu0,max_outputs=1 \ -device virtio-keyboard-device,id=vkbd0,display=vgpu0 \ -device virtio-mouse-device,id=vmouse0,display=vgpu0 \ ... ``` 注意:在ARM virt机器上,我们使用`virtio-gpu-device`等MMIO设备。 最后,关于屏幕类型(screen type)的进一步说明:virtio-gpu设备目前支持的是标准显示器,没有特定的屏幕类型(如CRT、LCD等)配置。但是,我们可以配置显示的分辨率等,这通常由客户机驱动和QEMU的显示后端共同决定。 因此,回答用户的问题: 1. 配置显示:通过QEMU的显示选项(如`-vnc`,`-display`)设置显示后端,并通过`display`属性将输入设备关联到virtio-gpu设备。 2. 屏幕类型:virtio-gpu设备通过`max_outputs`配置屏幕数量(多显示器),但每个屏幕的类型是标准显示器。 3. MMIO或PCI设置:通过选择不同的设备类型(`-device virtio-*-pci`或`-device virtio-*-device`)来实现。 注意:在配置时,确保QEMU版本支持这些设备(较新的QEMU版本通常支持)。 如果用户需要更具体的配置,可能需要根据实际需求调整。</think>### 配置 vdev virtio-input 创建虚拟输入设备接口 在 QEMU/KVM 虚拟化环境中,`virtio-input` 设备用于创建虚拟输入设备(如键盘、鼠标)。以下是完整配置指南,涵盖显示设置、屏幕类型和 MMIO/PCI 配置: #### 1. **基本 virtio-input 设备创建** ```bash qemu-system-x86_64 \ -device virtio-keyboard-pci,id=vkbd \ -device virtio-mouse-pci,id=vmouse \ ... ``` - `virtio-keyboard-pci`:创建 PCI 虚拟键盘 - `virtio-mouse-pci`:创建 PCI 虚拟鼠标 - `id`:设备唯一标识符(后续关联显示用) #### 2. **显示与屏幕类型关联** 通过 `display` 属性将输入设备绑定到特定显示: ```bash -device virtio-gpu-pci,id=vgpu0 \ -device virtio-keyboard-pci,display=vgpu0 \ -device virtio-mouse-pci,display=vgpu0 ``` - `virtio-gpu-pci`:创建虚拟 GPU 设备 - `display=vgpu0`:将输入设备绑定到该 GPU 的显示输出 **多屏幕配置**: ```bash -device virtio-gpu-pci,id=vgpu0,max_outputs=2 \ -device virtio-keyboard-pci,display=vgpu0.0 \ # 绑定到屏幕1 -device virtio-mouse-pci,display=vgpu0.1 # 绑定到屏幕2 ``` - `max_outputs=2`:启用双屏显示 - `display=vgpu0.X`:指定目标屏幕索引 #### 3. **MMIO 与 PCI 配置选项 **PCI 模式(默认)**: ```bash -device virtio-keyboard-pci,addr=0x1A \ # 指定 PCI 地址 -display gtk,gl=on # 使用 GTK 显示 ``` - `addr`:手动分配 PCI 地址(避免冲突) **MMIO 模式(ARM 架构常用)**: ```bash -machine virt,gic-version=3 \ -device virtio-keyboard-device \ -device virtio-mouse-device \ -display none,gl=off # 禁用图形显示 ``` - `virtio-*-device`:使用 MMIO 而非 PCI - 适用于无 PCI 总线的架构(如 ARM) #### 4. **完整配置示例(双屏 PCI 模式)** ```bash qemu-system-x86_64 \ -machine q35,accel=kvm \ -device virtio-gpu-pci,id=vgpu0,max_outputs=2 \ -device virtio-keyboard-pci,id=vkbd,display=vgpu0.0 \ -device virtio-mouse-pci,id=vmouse,display=vgpu0.1 \ -vga none \ # 禁用传统 VGA -display sdl,gl=on # 使用 SDL 显示后端 ``` #### 关键注意事项: 1. **驱动依赖**:客户机需安装 `virtio-input` 驱动(Linux 内核 ≥ 4.0 内置) 2. **显示后端兼容性**: - GTK/SDL:支持多屏幕绑定 - VNC/Spice:需额外配置 `-device virtio-serial` 传输事件 3. **性能优化**: ```bash -global virtio-pci.disable-modern=off \ # 启用 Virtio 1.0+ -global virtio-pci.iommu_platform=on # 启用 IOMMU ``` 4. **调试命令**: ```bash monitor> info qtree # 查看设备拓扑 monitor> info mtree # 检查 MMIO/PCI 映射 ``` > 参考文档: > [QEMU 设备模型 - virtio-input](https://www.qemu.org/docs/master/system/devices/virtio-input.html) [^1] > [VirtIO 标准 - 输入设备扩展](https://docs.oasis-open.org/virtio/virtio/v1.1/csprd01/virtio-v1.1-csprd01.html#x1-5800005) [^2] --- ### 相关问题 1. 如何在 KVM 虚拟机中实现多显示器分别接收不同的输入设备事件? 2. virtio-input 设备在 ARM 架构和 x86 架构下的性能差异有哪些? 3. 如何通过 libvirt 配置 virtio-input 的 PCI 地址分配? 4. virtio-gpu 与 virtio-input 协同工作时的事件传输机制是什么? 5. 在无图形界面的服务器环境中,virtio-input 设备有哪些应用场景?
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
红包 添加红包
表情包 插入表情
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值