Missing Pen backends

[icculus]

• iOS can work with Apple Pencil (and probably nothing else, but that's okay), using the same UITouch interface we already use; you just check the event's "type" to see if it came from a pencil instead of a finger, then metrics like rotation and stuff will be available on the event. I have an Apple Pencil and an iPad here, so if I can coerce the thing to work with my developer account, I'll try to implement this.
• Android supports styluses (and apparently can also use a USB tablet plugged into the device in some cases). Most Chromebooks with touchscreens and some phones/tablets can use a standard pen interface called "USI 2.0" ... the pens themself aren't too expensive. If my tablet will work with something here, I'll give it a try. Android takes the same approach as iOS: you get the same MotionEvent, but it has extra info for stylus input.
• Emscripten offers pointer events, which are widely supported, but I haven't actually tried this, so it might be the APIs are available but never work with tablets on most systems...I'll find some example page and see what happens here.
• Windows has several APIs for this, but I'm pretty sure the widest supported is RealTimeStylus, which goes back to Windows XP Tablet Edition (APIs included by default for Vista and later). There's also WinTab, which is installed when installing Wacom's drivers on Windows, but I'd probably avoid this. Windows Ink is a Windows 10 thing and probably higher level than we want. I don't know what WinRT offers, but surely something.

Originally posted by @icculus in #9938 (comment)

• Emscripten
• iOS
• Android
• Windows Ink
• WinTab

[icculus]

PR #9938 is merging shortly, which gets the ABI handled, but these are other backends to be implemented in the future.

For now I'm putting this in the 3.2.0 milestone, but any number of these backends can bump to 3.x.

[icculus]

Emscripten support works as of b975bab!

[icculus]

And you can see it in your browser here, if you have a pen/stylus.

[madebr]

And you can see it in your browser here, if you have a pen/stylus.

I like how the SDL types/functions in the Source Code are hyperlinks to the wiki. That's awesome :)

[askmeaboutlo0m]

I work on Drawpile and a bit on Krita and was looking at this cursorily. Unfortunately I don't have any code I can point to for reference because it's all GPL and I don't have full ownership of it, but maybe some of this experience is still helpful, so I'll just spill it here in the hopes something in there being useful.

All Windows drawing applications I know of let you use both Windows Ink and Wintab because some tablets only work properly with one or the other. Windows Ink also has a bunch of undesirable behavior, like right-click emulation and weird pen feedback animations that the user has dig through several different settings apps and ancient control panel dialogs to disable. I've never heard of any drawing application using RealTimeStylus. Web browsers all use Windows Ink, for Wintab support there's only this third-party Chrome plugin.

On Android, you can plug in a Wacom tablet in via USB OTG, but it doesn't work on all devices. It'll also default to a ridiculous portrait mode that only uses a fraction of the tablet surface, but the events you get are the same as with a "real" pen, so probably good enough to develop against.

Also, SonarPen exists for Android and iOS, which uses a bespoke interface. I can't speak to the quality of the devices, but they've offered to send me one of their devices before and have done some legwork of their own to implement it for Krita, so you may get something out of them if you ask.

There's a comment in the Emscripten implementation wondering if button 5 is really the eraser: yep, that's correct. And you don't get any information whether the pen tip or eraser tip is in use until it's pressed down, which means the application can't switch tools depending on the proximity events. Or, well, I guess in SDL's case it would be pen motion events, since the proximity events don't tell you the pen state. But may be worth pointing out this platform quirk in the documentation.

From my work in implementing Emscripten tablet support in Qt, bringing the pen near the device while a touch operation is in progress implicitly cancels the touch, but you don't get any events to that effect. Unsure if this is because of how Qt deals with it or if it's the browser behaving like this, but may be something to test out.

Also on Emscripten, the Apple Pencil on iPad and iPhone seems to report ridiculously low pressure values compared to other pens. I don't think there's anything SDL can really do about this except maybe document it, in Drawpile we look at the user agent and set the pressure curve to be really really steep if it smells like such a device.

Wayland on Gnome and KDE has some really strange multi-cursor implementation with tablets that breaks the cursors on all Qt applications and you end up with no cursor or a default shape respectively. Not sure if SDL handles this properly or how to fix it, I just know the questions we get about it.

[Susko3]

For Windows 8+, you probably want win32 pointer input. I think if you try supporting pointer inputs for pens, you might be forced to use it for touch as well.

[askmeaboutlo0m]

"Win32 pointer input" is the Windows Ink I talked about in my comment. Wacom tablets and various others let you toggle it using a checkbox with that title in their driver settings.

[icculus]

Initial Android support is in (thanks, @hwsmm!).

[icculus]

Windows support is in, we're good to go on all reasonable platforms now!

[slouken]

Woo!!

[maia-s]

Can you please add wintab support for Windows too? Windows Ink is often turned off by artists because of the undesirable behaviour mentioned in #10516 (comment)

[icculus]

Probably not before 3.2.0 ships, but afterwards we can.

[icculus]

(I'll leave this in the milestone for now though, just in case there's a moment to get to it.)