Hey Luke, have you considered use of facial gestures to control tempo, trigger foot-out to stop, two-feet to continue, etc?
1 Like
I must say, I have not! Spoken commands (“Stop!” “Speed up” etc.) have been suggested, and I have explored that and ran into technical challenges and a “check back with the tech in a couple years” decision. Facial “gestures” have not occurred to me, nor been suggested before, but it’s a clever idea. It’d require constant analysis of your camera, which I’d want to do fully local and on-device for privacy reasons… but it could be doable.
Got a bunch of more pressing updates to work on but I’ll keep this on the list of things to explore in the future!
I’ve been playing around the accessibility features in iOS but can’t figure out how to apply it to my apps. The forScore app has some premium functionality that seems to work well for page turns. Not sure how it would work for me in a practical sense. From the facial contortions in videos of me playing my banjo you might think I was in severe pain. I’m really not. Those listening to me may be. With Strum Machine I can easily play 2 plus hours without a break.
I think using facial expressions is…well… disgusting, as well as frightening. Our local Kroger store uses facial recognition cameras in the shopping aisles, and I have bought a whole-head black see-through covering for when I go there. I will also use this covering when I go to political protests. Facial recognition technology is used by the various law enforcement bodies, for good or ill. It is not something to have fun with.
I share all of these concerns with facial recognition technology, @MarianDrake. It’s scary stuff.
To be clear, if I were to do anything with this technology in Strum Machine (or voice recognition or any other “smart” features) it would only be if I could be 100% sure no data left your device or could be used for any other purposes.
And to be honest, I think the odds of this happening are low… but I’m not against looking into it further if I can do it in this complete-privacy way.
In the meantime, though, for hands-free controlling of Strum Machine I think a Bluetooth foot pedal is the way to go. I’ll be adding better support for these (customizable commands for multi-switch pedals) in the future as well.
Hi Luke, Than so much for your caring reply. As for the remote possibility of being “100% sure” that this sort of technology not be used for nefarious purposes, I personally believe that could never, ever be possible. And why? Because what might 100% protected when first implemented by Strum Machine or other organizations who might want to do this, in a rather short period of time after implementing a “safe” way so use it would soon be technologically compromised by other institutions interested in getting at users’ personal data. (All one has to do for an excellent example of how technology can be manipulated is the political and personal misuse of AI!)
If I were to implement something like facial gesture recognition, there could be a secure way to do it:
- Only use open-source code so I can deeply inspect and audit what’s going on under the hood
- Do all processing on the device; ensure nothing goes out onto the internet
- Do not store any data on (or off) the device
For example, I do something similar with voice dictation on my computer for personal use; I use open-source software that does the transcription locally. Nothing leaves the device, no other people or organizations are involved.
This is what I mean by being “100% sure” – not trusting other organizations to “do the right thing”, but setting things up in Strum Machine in such a way that it’s impossible for other organizations to do nefarious things.
I suppose the weak link there is whether users would trust that I am indeed doing what I say. But I have no incentive to do otherwise! I don’t want users’ personal data, and I care about their right to privacy. I know it’s rather trite for a company to say they care about users’ privacy, but Strum Machine is arguably not much of a company, since it’s basically just me here. 
Anyway, if this feature ever sees the light of day (big if) it will be an opt-in thing and off by default for sure.