简介
多模输入系统主要用于接收按键，触摸等输入事件，并且会对这些原始输入事件进行处理，之后再对这些事件进行派发。同时多模输入系统还提供了注入事件的接口，应用可以通过调用这个接口产生输入事件，然后将该输入事件注入到输入系统中进行处理。
输入系统框架
多模输入系统主要是由InputManagerService, InputEventHub, InputEventDistributer来负责处理的。InputManagerService会启动InputEventHub，并且会通过创建子线程的方式来创建InputEventDistributer。当底层传来按键或触摸事件的时候，InputEventHub就会进行读取，并且会对这些原始的输入事件进行处理，处理完后会交给InputEventDistributer进行派发。InputEventDistributer又会通过InputEventClientProxy进行IPC交互的方式发给应用端。
多模输入系统事件派发流程
事件派发流程图
源码分析
下面就对多模输入系统事件派发流程的源码进行分析。
InputManagerService
\foundation\graphic\wms\services\wms\wms.cpp
- int main()
- {
- DEBUG_PERFORMANCE_REGISTER_SIG();
- OHOS::HiFbdevInit();
- OHOS::GfxEngines::GetInstance()->InitDriver();
- HOS_SystemInit();
- OHOS::InputManagerService::GetInstance()->Run();
- while (1) {
- DEBUG_PERFORMANCE_PRINT_RESULT();
- OHOS::LiteWM::GetInstance()->MainTaskHandler();
- usleep(WMS_MAIN_TASK_PERIOD_IN_US);
- }
- }
InputManagerService的启动是在WMS的main函数中通过InputManagerService::GetInstance()->Run()执行的。
\foundation\graphic\wms\services\ims\input_manager_service.cpp
- void InputManagerService::Run()
- {
- hub_ = InputEventHub::GetInstance();
- hub_->RegisterReadCallback(ReadCallback);
- hub_->SetUp();
- distributerThreadCreated_ = pthread_create(&distributerThread_, nullptr, Distribute, nullptr);
- if (!distributerThreadCreated_) {
- pthread_detach(distributerThread_);
- }
- }
在InputManagerService::Run()中首先会创建InputEventHub的对象并通过RegisterReadCallback来注册InputEventHub的回调，然后通过SetUp来启动InputEventHub, InputEventHub主要是用于对底层原始输入事件的读取和处理，该函数的最后会创建distributerThread子线程，用于对输入事件的派发。
InputEventHub
\foundation\graphic\wms\services\ims\input_event_hub.cpp
- void InputEventHub::SetUp()
- {
- int32_t ret = GetInputInterface(&inputInterface_);
- if (ret != INPUT_SUCCESS) {
- GRAPHIC_LOGE("get input driver interface failed!");
- return;
- }
- uint8_t num = ScanInputDevice();
- if (num == 0) {
- GRAPHIC_LOGE("There is no device!");
- return;
- }
- for (uint8_t i = 0; i < num; i++) {
- if (inputInterface_ == nullptr || inputInterface_->iInputManager == nullptr) {
- GRAPHIC_LOGE("input interface or input manager is nullptr, open device failed!");
- return;
- }
- ret = inputInterface_->iInputManager->OpenInputDevice(mountDevIndex_[i]);
- if (ret == INPUT_SUCCESS && inputInterface_->iInputReporter != nullptr) {
- callback_.EventPkgCallback = EventCallback;
- ret = inputInterface_->iInputReporter->RegisterReportCallback(mountDevIndex_[i], &callback_);
- if (ret != INPUT_SUCCESS) {
- GRAPHIC_LOGE("device dose not exist, can't register callback to it!");
- return;
- }
- openDev_ = openDev_ | (1 << i);
- }
- }
- }
在这个函数中InputEventHub主要的工作就是通过调用驱动层的OpenInputDevice来打开输入设备，并且会将EventCallback的回调函数通过驱动层的RegisterReportCallback进行注册。当底层有事件传递上来，EventCallback就会被调用。OpenInputDevice和RegisterReportCallback具体实现分别是在drivers/peripheral/input/hal/src/input_manager.c和drivers/peripheral/input/hal/src/input_reporter.c中。
\foundation\graphic\wms\services\ims\input_event_hub.cpp
- void InputEventHub::EventCallback(const EventPackage **pkgs, uint32_t count, uint32_t devIndex)
- {
- if (pkgs == nullptr || readCallback_ == nullptr || count == 0) {
- return;
- }
- RawEvent& data = InputEventHub::GetInstance()->data_;
- for (uint32_t i = 0; i < count; i++) {
- if (pkgs[i]->type == EV_REL) {
- data.type = InputDevType::INDEV_TYPE_MOUSE;
- if (pkgs[i]->code == REL_X)
- data.x += pkgs[i]->value;
- else if (pkgs[i]->code == REL_Y)
- data.y += pkgs[i]->value;
- } else if (pkgs[i]->type == EV_ABS) {
- data.type = InputDevType::INDEV_TYPE_TOUCH;
- if (pkgs[i]->code == ABS_MT_POSITION_X)
- data.x = pkgs[i]->value;
- else if (pkgs[i]->code == ABS_MT_POSITION_Y)
- data.y = pkgs[i]->value;
- } else if (pkgs[i]->type == EV_KEY) {
- if (pkgs[i]->code == BTN_MOUSE || pkgs[i]->code == BTN_TOUCH) {
- if (pkgs[i]->value == 0)
- data.state = 0;
- else if (pkgs[i]->value == 1)
- data.state = 1;
- }
- } else if (pkgs[i]->type == EV_SYN) {
- if (pkgs[i]->code == SYN_REPORT) {
- break;
- }
- }
- }
- readCallback_(&data);
- }
当底层有输入事件上来的话，EventCallback就会被调用，在这个函数里会通过EventPackage->type来判断输入事件的类型，其中
EV_REL是相对坐标的输入事件，比如轨迹球，鼠标事件
EV_ABS是绝对坐标的输入事件，比如触屏触摸事件
EV_KEY是按键输入事件，比如设备上的物理按键的点击事件
EV_SYN是Motion的一系列动作结束标志位
如果是鼠标事件，会将相对坐标值放入到data.x和data.y中，如果是触屏触摸事件，会将在触屏上触摸的坐标位置放入到data.x和data.y中，如果是按键事件会将按键的点击状态放入到data.state中。
处理完输入事件后，会将数据放入到data中，并通过readCallback传给InputManagerService进行处理，之后就会调用InputManagerService::ReadCallback。
\foundation\graphic\wms\services\ims\input_manager_service.cpp
- void InputManagerService::ReadCallback(const RawEvent* event)
- {
- if (event == nullptr) {
- return;
- }
- pthread_mutex_lock(&lock_);
- while (eventQueue_.size() == MAX_EVENT_SIZE) {
- pthread_cond_wait(&nonFull_, &lock_);
- }
- // push events into queue
- eventQueue_.push(event[0]);
- pthread_mutex_unlock(&lock_);
- pthread_cond_signal(&nonEmpty_);
- }
- void* InputManagerService::Distribute(void* args)
- {
- GRAPHIC_LOGI("InputManagerService::Distribute Ready to read distribute!");
- while (true) {
- pthread_mutex_lock(&lock_);
- while (eventQueue_.size() == 0) {
- pthread_cond_wait(&nonEmpty_, &lock_);
- }
- // pop events from queue
- RawEvent events[MAX_INPUT_DEVICE_NUM];
- int32_t len = (eventQueue_.size() > MAX_EVENT_SIZE) ? MAX_EVENT_SIZE : eventQueue_.size();
- for (int32_t i = 0; i < len; i++) {
- events[i] = eventQueue_.front();
- eventQueue_.pop();
- }
- distributer_.Distribute(events, len);
- pthread_mutex_unlock(&lock_);
- pthread_cond_signal(&nonFull_);
- }
- return nullptr;
- }
ReadCallback这个函数首先会判断eventQueue这个事件队列里事件数量是否达到最大数量，如果达到最大数量该线程就一直等待，否则就会把该事件放到eventQueue这个事件队列里，并且同时也会发出nonEmpty的signal, 来让Distribute中的线程停止等待。
Distribute函数中，当eventQueue队列里没有事件的时候，就会一直等待，当有事件来的时候就会停止线程等待，然后会遍历整个eventQueue这个队列，把每个事件获取出来后放入到events这个数组中，并做为参数放入到InputEventDistributer::Distribute中进行事件的派发。
InputEventDistributer
\foundation\graphic\wms\services\ims\input_event_distributer.cpp
- void InputEventDistributer::Distribute(const RawEvent* events, int32_t size)
- {
- for (int32_t i = 0; i < size; i++) {
- for (auto listener : rawEventListeners_) {
- if (listener != nullptr) {
- listener->OnRawEvent(events[i]);
- }
- }
- }
- }
这个函数比较简单，主要就是遍历所有的InputEventClientProxy, 并且调用各自的onRawEvent进行实际的派发工作。
InputEventClientProxy
\foundation\graphic\wms\services\ims\input_event_client_proxy.cpp
- void InputEventClientProxy::OnRawEvent(const RawEvent& event)
- {
- IpcIo io;
- uint8_t tmpData[IMS_DEFAULT_IPC_SIZE];
- IpcIoInit(&io, tmpData, IMS_DEFAULT_IPC_SIZE, 1);
- IpcIoPushFlatObj(&io, static_cast<const void*>(&event), sizeof(RawEvent));
- pthread_mutex_lock(&lock_);
- std::map<pid_t, ClientInfo>::iterator it;
- for (it = clientInfoMap_.begin(); it != clientInfoMap_.end(); it++) {
- if (it->second.alwaysInvoke || (event.state != lastState_)) {
- SendRequest(nullptr, it->second.svc, 0, &io, nullptr, LITEIPC_FLAG_ONEWAY, nullptr);
- }
- }
- lastState_ = event.state;
- pthread_mutex_unlock(&lock_);
- }
这个函数主要就是通过ipc的交互方式把输入事件传给应用端。
到此整个多模输入系统的事件派发流程就结束了。
多模输入系统接口说明
模块
- /foundation/multimodalinput/input
- ├── common # 公共代码
- ├── interfaces # 对外接口存放目录
- │ └── native # 对外native层接口存放目录
- │ └── innerkits # 对系统内部子系统提供native层接口存放目录
- ├── service # 服务框架代码
- ├── sa_profile # 服务启动配置文件
- ├── uinput # 输入事件注入模块
通过每个目录下的.gn文件可以看到每个目录下的模块都对应动态库
\interfaces\native\innerkits\event下的文件编出来的是mmi_event.so
\interfaces\native\innerkits\napi 下的文件编出来的是injecteventhandler.so
\interfaces\native\innerkits\proxy 下的文件编出来的是libmultimodalinput_proxy.so
\service 下的文件编出来的是libmultimodalinput_service.so
\uinput 下的文件编出来的是mmi_uinject.so
接口
多模输入目前提供的接口为事件注入接口，该接口目前仅对系统应用开放。
JS接口
InJectEventHandler是处理注入事件类。
\applications\standard\systemui\navigationBar\src\main\js\default\pages\backKey\backKey.js
- export default {
- /**
- * User start touching the back button
- */
- backTouchStart() {
- mLog.showInfo(TAG, `back touch start`);
- res = input.injectEventSync({
- isPressed: true,
- keyCode: 2,
- keyDownDuration: 1
- });
- mLog.showInfo(TAG, `injectEventHandler injectEventSync down res: ${res}`);
- },
- /**
- * User stop touching the back button
- * Trigger "Back" event
- */
- backTouchEnd() {
- mLog.showInfo(TAG, `back touch end and injectEventHandler injectEventSync`);
- res = input.injectEventSync({
- isPressed: false,
- keyCode: 2,
- keyDownDuration: 1
- });
- mLog.showInfo(TAG, `injectEventHandler injectEventSync up res: ${res}`);
- }
- }
可以从openharmony systemui的navigationbar的源码中看到, 当点击navigationbar的back键的时候，就会调用js的接口函数injectEventSync，并传入三个参数，其中
isPress: 按键的状态，true表示down, false表示up
keyCode：键值码，2表示back事件
keyDownDuration：按键按下到抬起之间的时长，单位ms，1表示1ms
C++接口
系统内部接口
在\interfaces\native\innerkits\events\include下的头文件都定义了各自对内部系统调用的口。
KeyEvent的主要接口
KeyBoardEvent的主要接口
ManipulationEvent的主要接口
MmiPoint的主要接口
MouseEvent的主要接口
MultimodalEvent的主要接口
StylusEvent的主要接口
TouchEvent的主要接口
InjectEvent的实现逻辑
\foundation\multimodalinput\input\interfaces\native\innerkits\napi\src\key_event_handler.cpp
- static napi_value InjectEventSync(napi_env env, napi_callback_info info)
- {
- size_t argc = 2;
- napi_value args[2] = { 0 };
- napi_value thisArg = nullptr;
- void* data = nullptr;
- NAPI_CALL(env, napi_get_cb_info(env, info, &argc, args, &thisArg, &data));
- napi_value eventObject = args[0];
- int32_t ret = IsMatchType(eventObject, napi_object, env);
- if (ret) {
- return GetNapiInt32_t(ret, env);
- }
- napi_value isPressed, keyCode, keyDownDuration;
- napi_get_named_property(env, eventObject, "isPressed", &isPressed);
- napi_get_named_property(env, eventObject, "keyDownDuration", &keyDownDuration);
- napi_get_named_property(env, eventObject, "keyCode", &keyCode);
- if (IsMatchType(isPressed, napi_boolean, env) || IsMatchType(keyCode, napi_number, env)
- || IsMatchType(keyDownDuration, napi_number, env)) {
- return GetNapiInt32_t(-1, env);
- }
- OHOS::KeyProperty keyProperty = {
- .isPressed = GetCppBool(isPressed, env),
- .keyCode = GetCppInt32_t(keyCode, env),
- .keyDownDuration = GetCppInt32_t(keyDownDuration, env),
- };
- OHOS::MultimodalProperty multimodalProperty {
- .highLevelEvent = 1,
- .uuid = "11111",
- .sourceType = 1,
- .occurredTime = 1,
- .deviceId = "11111",
- .inputDeviceId = 1,
- .isHighLevelEvent = true,
- };
- OHOS::sptr<OHOS::KeyEvent> event = new OHOS::KeyEvent();
- if (!event) {
- return GetNapiInt32_t(-1, env);
- }
- event->Initialize(multimodalProperty, keyProperty);
- std::shared_ptr<OHOS::InjectManager> injectManager = OHOS::InjectManager::GetInstance();
- bool isSucceed = injectManager->InjectEvent(event);
- if (!isSucceed) {
- return GetNapiInt32_t(-1, env);
- }
- return GetNapiInt32_t(0, env);
- }
在key_event_handler.cpp中实现了InjectEventSync这个接口，通过NAPI获得应用端的isPressed,KeyDownDuration,KeyCode这三个数值，并将这三个参数放入到KeyProperty这个结构体中。然后调用KeyEvent的Initialize，将KeyProperty封装到KeyEvent中，最后再调用InjectManager的InjectEvent。
\foundation\multimodalinput\input\interfaces\native\innerkits\proxy\src\inject_manager.cpp
- bool InjectManager::InjectEvent(const sptr<MultimodalEvent> event)
- {
- std::lock_guard<std::mutex> guard(lock_);
- if (!multimodalInputService_) {
- return false;
- }
- int32_t result = multimodalInputService_->InjectEvent(event);
- if (result == 0) {
- return true;
- }
- MMI_LOGI("inject failed");
- return false;
- }
foundation\multimodalinput\input\interfaces\native\innerkits\proxy\include\inject_manager.h
- sptr<IMultimodalInputService> multimodalInputService_{nullptr};
multimodalInputService_->InjectEvent其实是一个IPC进程间调用，这会调用到客户端的MultimodalInputServiceProxy的InjectEvent。
foundation\multimodalinput\input\interfaces\native\innerkits\proxy\src\multimodal_input_service_proxy.cpp
- int32_t MultimodalInputServiceProxy::InjectEvent(const sptr<MultimodalEvent> &event)
- {
- MessageParcel data;
- MessageParcel reply;
- MessageOption option(MessageOption::TF_ASYNC);
- if (!data.WriteInterfaceToken(MultimodalInputServiceProxy::GetDescriptor())) {
- HiLog::Error(LABEL, "write descriptor fail");
- return ERR_INVALID_VALUE;
- }
- if (!data.WriteInt32(MultimodalEvent::KEYBOARD)) {
- HiLog::Error(LABEL, "write descriptor fail");
- return ERR_INVALID_VALUE;
- }
- if (!data.WriteParcelable(event)) {
- HiLog::Error(LABEL, "inject event fail, write event error");
- return ERR_INVALID_VALUE;
- }
- int error = Remote()->SendRequest(INJECT_EVENT, data, reply, option);
- if (error != ERR_NONE) {
- HiLog::Error(LABEL, "inject event fail, error: %{public}d", error);
- }
- return error;
- }
在MultimodalInputServiceProxy::InjectEvent会通过SendRequest向服务端MultimodalInputServiceStub发送数据。
foundation\multimodalinput\input\service\src\multimodal_input_service_stub.cpp
- int MultimodalInputServiceStub::OnRemoteRequest(
- uint32_t code, MessageParcel &data, MessageParcel &reply, MessageOption &option)
- {
- MMI_LOGD("OnReceived, cmd = %{public}u", code);
- if (!IsPermissionValid()) {
- MMI_LOGE("calling app not acquired multimodal permission");
- return MMI_PERMISSION_ERR;
- }
- std::u16string myDescripter = MultimodalInputServiceStub::GetDescriptor();
- std::u16string remoteDescripter = data.ReadInterfaceToken();
- if (myDescripter != remoteDescripter) {
- MMI_LOGE("descriptor checked fail");
- return MMI_BAD_TYPE;
- }
- switch (code) {
- case INJECT_EVENT: {
- int32_t type = data.ReadInt32();
- if (type == MultimodalEvent::KEYBOARD) {
- sptr<MultimodalEvent> event = data.ReadParcelable<KeyEvent>();
- return InjectEvent(event);
- }
- MMI_LOGE("recv bad type %{public}d", type);
- return MMI_BAD_TYPE;
- }
- default: {
- MMI_LOGE("default case, need check");
- return IPCObjectStub::OnRemoteRequest(code, data, reply, option);
- }
- }
- }
通过sendRequest将数据发送之后，服务端的MultimodalInputServiceStub的OnRemoteRequest就会被调用，最终会调用MultimodaInputService的InjectEvent。
\foundation\multimodalinput\input\service\src\multimodal_input_service.cpp
- int32_t MultimodalInputService::InjectEvent(const sptr<MultimodalEvent> &event)
- {
- KeyEvent *eventPtr = reinterpret_cast<KeyEvent*>(event.GetRefPtr());
- int keycode = eventPtr->GetKeyCode();
- int state = 0;
- if (eventPtr->IsKeyDown()) {
- state = 1;
- } else {
- state = 0;
- }
- MMIS::KeyboardInject &inject = OHOS::MMIS::KeyboardInject::GetInstance();
- MMI_LOGD("InjectEvent keycode %{public}d, state %{public}d", keycode, state);
- inject.InjectKeyEvent(keycode, state);
- return 0;
- }
MultimodaInputService的InjectEvent实际上会调用KeyboardInject的InjectKeyEvent，从函数的实现来看，目前只使用了KeyboardInject，也就是说目前只支持键盘事件的注入。
\foundation\multimodalinput\input\uinput\keyboard_inject.cpp
- void KeyboardInject::InjectKeyEvent(uint16_t code, uint32_t value) const
- {
- std::lock_guard<std::mutex> keyboardLock(mutex_);
- auto it = keyCodeMap_.find(code);
- if (it == keyCodeMap_.end()) {
- return;
- }
- InjectInputEvent injectInputEvent = {injectThread_->KEYBOARD_DEVICE_ID, EV_KEY, it->second, value};
- injectThread_->WaitFunc(injectInputEvent);
- InjectInputEvent injectInputSync = {injectThread_->KEYBOARD_DEVICE_ID, EV_SYN, SYN_REPORT, 0};
- injectThread_->WaitFunc(injectInputSync);
- }
在InjectKeyEvent中会通过InjectInputEvent的WaitFunc将注入事件继续向下注入。
\foundation\multimodalinput\input\uinput\inject_thread.cpp
- void InjectThread::InjectFunc() const
- {
- std::unique_lock<std::mutex> uniqueLock(mutex_);
- while (true) {
- conditionVariable_.wait(uniqueLock);
- while (injectQueue_.size() > 0) {
- if (injectQueue_[0].deviceId == TOUCH_SCREEN_DEVICE_ID) {
- g_pTouchScreen->EmitEvent(injectQueue_[0].type, injectQueue_[0].code, injectQueue_[0].value);
- } else if (injectQueue_[0].deviceId == KEYBOARD_DEVICE_ID) {
- g_pKeyboard->EmitEvent(injectQueue_[0].type, injectQueue_[0].code, injectQueue_[0].value);
- }
- injectQueue_.erase(injectQueue_.begin());
- }
- }
- }
- void InjectThread::WaitFunc(InjectInputEvent injectInputEvent) const
- {
- std::lock_guard<std::mutex> lockGuard(mutex_);
- injectQueue_.push_back(injectInputEvent);
- conditionVariable_.notify_one();
- }
在WaitFunc中会将injectInputEvent放入到injectQueue这个队列中，这个队列是用来存放injectInputEvent的，并且通过notify_one来唤醒InjectThread，由于目前只支持键盘类型事件的注入，所有只会调用g_pKeyboard->EmitEven()，g_pKeyboard是VirtualKeyboard的对象，VirtualKeyboard又继承自VirtualDevice，因此最终会调用VirtualKeyboard的EmitEvent。
foundation\multimodalinput\input\uinput\virtual_device.cpp
- bool VirtualDevice::EmitEvent(uint16_t type, uint16_t code, uint32_t value) const
- {
- struct input_event event {};
- event.type = type;
- event.code = code;
- event.value = value;
- #ifndef __MUSL__
- gettimeofday(&event.time, NULL);
- #endif
- if (write(fd_, &event, sizeof(event)) < static_cast<ssize_t>(sizeof(event))) {
- HiLog::Error(LABEL, "Event write failed %{public}s aborting", __func__);
- return false;
- }
- return true;
- }
- bool VirtualDevice::SetUp()
- {
- fd_ = open("/dev/uinput", O_WRONLY | O_NONBLOCK);
在该函数中会将这个注入事件写入到文件描述符为fd_的设备文件中，从SetUp的函数中可以看出实际是写入到/dev/uinput这个设备文件中。
到此多模输入系统接口的介绍以及InjectEvent整个注入事件的流程就结束了。
总结
通过本文的学习可以了解多模输入系统事件派发的流程，以及多模输入系统的接口和注入事件的流程，结合以上的源码分析会对多模输入子系统会有更深入的理解。
