I’ve been thinking lately about the inevitable Apple based AI assistant, which will obviously be some kind of Siri based box. This comes from having used Siri for a number of years and only recently having used Amazon Echo’s Alexa. Having set up HomeKit and Alexa and getting it to work with a number of systems at home such as Harmony Hub and Sonos and August involved a number of glue systems such as Yonomi and Homebridge and Home-Assistant and Haaska. None of these are simple systems to set up, but all seem like something that should already be in both the HomeKit and Alexa ecosystems. They are not connected because of feuding between Apple and Amazon, and while that conflict is natural given the early stages of home AIs and the aggressive competitive nature of these two companies, it is frustrating for home users that just want to use a bunch of cool products together.

What sort of things would we want from an Apple based home assistant? Taking the best features from the existing services, I imagine that it would look something like this:

• Disembodied assistant – the benefit of this is that I can just talk to it without deciphering the visual feedback, which is where something like Alexa is a win. No screen but maybe some LEDs to quickly see some kind of visual feedback.
• Compact array of speakers and a microphone- the Echo devices are an amazing always-on tool dedicated to listening for you and speaking back to you, another Alexa innovation.
• No battery necessary – it is always plugged in. Maybe if surviving a power blip is important something small can be included to keep the logic running, but having a battery means a shorter lifespan. Like Apple TV, this is something that won’t be swapped out yearly. BTW, the Evo is a neat battery for Amazon Echo Dot.
• Needs to be cheap – the Echo Dot is all I need in a device and is only $50. Amazing.
• App and web status – the Alexa app lets you get a view of all the devices that the system knows about. Home-assistant gives a very nice web based interface for showing all your devices and their status. The Home app, although it needs more automation complexity, gives a good view of rooms and devices and automations.

The Apple product that currently comes closest in form and function to this description is the Apple Watch. Given a tear-down of the Apple Watch, you can ditch the battery, the compact form, the display, the taptic engine, the rugged shell, the ingress protection, gps capability, barometric pressure sensor, the health gathering (heart rate sensor and motion sensors) – you are just left with the S2 SiP (a very thin and tiny board) and a speaker. Technically, you could even just go with the S1P SiP since the onboard GPS is not needed. That also still leaves you with very good bluetooth and wifi in a very tiny package. While this wafer thin system is not a significant electronic component, it is the brains of the whole operation that  is capable of taking on the task of just being present to serve as a Siri proxy. Given the S2 SiP as a starting point, what would you then need to add to it to make it a real product? My list would be:

• Improved bluetooth – the W1 chips in the AirPods have amazing range, I’d expect the same for interacting with many HomeKit devices
• improved wifi – this device needs to be ever present and ready, the wifi in the Apple Watch may not be strong enough
• Improved speakers and microphones – the Apple Watch speaker and microphone are great for personal use and the few dings and bings that it does, but I would not use that to properly interact from across the room
• larger case – appearance is everything in these devices. If you sell a device that looks like a poker chip and expect to charge more than $25 for it, that could be a value perception problem. The size of the Apple TV is just right and would differentiate it from the Echo devices. There should be some LED type display that gives visual feedback that it heard you, but not something you need to decipher to figure out the response – what about a touchbar style display that just shows the Siri response wave.

Given that Apple has promoted Siri as the assistant that is always with you, it would be quite a turn for Apple to instead go with a stationary device, but it’s not the first time they have pivoted on a successful product. There are also some challenges with this.

• privacy – what can Siri be asked on this device that someone else should not know? What is being missed if not locking it down by not having any interface other than voice? Does any data live on this at all? What is allowed to be shared? This may be where circles of friendship (circles of privacy) could come into play and all “open” contacts lists and calendars and communication are centralized in this device.
• multiple users – will everyone in the family log into this? would you just have another iCloud account that joins Family Sharing as an AI member? Does everyone join this as a bluetooth device which will track them? What about guests (would be creepy to ask them to register)? This would provide presence. This would also require voice recognition as to who is issuing commands such as if you are sending a message – this is much easier with personal devices, but this is more of a device available to all.
• storage – is this going to eat into the little itty bitty 5GB total that Apple currently doles out to iCloud users?
• management interface – there should be some kind of interface to see what the device knows. I’m not asking for a full content graph diagram but even a static list in iCloud.com would be good. The voice interface is not enough for everything. The Alexa app provides some post-mortem of what the device heard which you provide feedback to rate how it’s doing as Amazon continues to develop the system.
• Better Siri – the Siri in Apple Watch is somewhat watered down version of what is in the iPhone. Given the bigger volume available in a bigger shell, Siri could potentially be more powerful in this device than in the iPhone itself. A lot of the jokes people make about Siri apply to Alexa and any other assistants as well, none are perfect. If you turn your head momentarily while talking, they all hiccup in interpreting your wishes. As humans, we just reply “huh?” but with AIs we seem to expect perfection even from our imperfect input. The technology still needs to evolve.
• HomeKit is currently in its infancy – Open HomeKit to more than just hardware manufacturers. All their apps just look like the Home app with a different skin and are no more capable. Why can’t HomeKit and Alexa just get along? Talk to non-HomeKit devices. Heck, talk to IFTTT and Workflow! Give us plugins. Some very capable apps like Yonomi and Stringify are waiting to trigger your HomeKit devices!

That sounds like a complete product to me, but Apple has a tendency to premium price its products, however to compete with a $50 full featured hockey puck, it cannot be more than $99. Even still, that price will be tough to stay at. Consider the AirPods which are compact and difficult to manufacture, are not a stand-alone product, and are still $159. At $99 I would not only buy one, but probably 3 or 4 of them to have them around the home for the family to use.

It’s that time of year where we prepare for new Apple announcements regarding what will come in the next version of iOS. Assuming the next version of iOS will be called 11, I’ve reviewed my posts from a year ago and came up with this list. What do you want to see in iOS 11?

—
Calendar 

• “shared by Reminders” option

TV

• multiple providers
• Comcast!!

Photos

• direct support for gifs
• TouchID for hidden photos
• Further AR in camera as demonstrated in portrait mode

Maps

• Waypoints
• Lane guidance
• Driver vs passenger activity
• Integrate directly with Find My Friends
• Checkin status

Contacts

• group management
• relationship circles including Family Sharing
• Include more apps like Line, Kik and WhatsApp

iCloud

• file sharing, a la Dropbox
• storage per device vs per account to allow backups

Keyboard

• picker vs current Next Keyboard functionality (similar to iMessage apps)
• more functionality like iMessage apps

iMessage

• plugins to centralize messaging for apps like WhatsApp and Kik 
• continue working on top of other apps via chat heads style
• Combine with phone and FaceTime
• Allow other video apps to integrate

Home screen

• Widgets among icons
• Make all home screens like Today screen
• More 3D Touch functionality – delete?

Apple News

• real RSS reader subscribe functionality
• Missed articles, not just too articles

Music

• Saving MP3s direct to Music
• Star ratings

Document management 

• Across device among apps
• including attachments management 

Improved password management

• Dedicated password app
• plugins for 1Password and LastPass?

HomeKit 

• further complex automation – time randomization 
• more participation beyond hardware manufacturers
• Home presence tracking 

Siri

• cooperative among devices
• iMessage integration
• Dedicate Siri-in-a-box device

Automation

• URL scheme expansion now that Workflow is purchased. 
• Allow more automated app launching

Control Center

• Placement customization
• Select icons to swap out 
• Bigger drawer

Multitasking

• Side by side on Plus devices 
• Slide out

Appearance

• Dark mode / themes
• Game Center banner gone, please!
• Volume indicator location – allow customization similar to banners – make a banner that temporarily displays to show volume decrease. Stop overlaying on app.
• Color folder tagging
• Notification Center sort by app

Misc

• Multiple timers
• Apple Pencil support iPhone 
• Option Share sheet as list like document picker
• Wifi priority

Ever since professional asshat Kellyanne Conway coined the term Alternate Facts, a whole new avenue of truths has been opened that nobody but complete liars ever realized could be mined. I present to you the last couple of weeks’ worth of #AFOTD images I’ve produced. You can also find these on my Twitter feed https://www.twitter.com/shokk

This post concludes my list of  iOS 10 features I want to see. You can see all the parts by starting here. (Part 2, part 3, part 4, part 5, part 6, part 7, part 8, part 9, and part 10)

Miscellaneous and Someday/Maybe Dream Items

By now iOS should be able to handle NFC reading in a secure sandboxed fashion. There is a lot of utility in getting URLs via NFC as handlily as we do with QR codes. By way of example, at my work place there are paintings hung with little cards descriptive cards next to them. The cards can be scanned with a mobile device to get more info on the painting. I can’t do that with an iPhone without some stupid case contraption connected. That’s a ridiculous embarrassment.

This one is for the power users: allow even more URL schemes for stock apps. Those schemes are a hidden treasury trove of functionality and one of the most powerful ways of working with iOS. The vast majority of users do not know about URL schemes but it is a key feature of some of the best productivity apps in the App Store. Apple should expand on this even further to make our devices take advantage of even more automation. I already use it with Workflow to send template emails when certain events happen, and writing Drafts 4 actions.

Automatic launch actions based on changed conditions: wifi, Bluetooth, location. rimilar to what Siri wants to do but in a notification too. This may be too niche and too prone to rapid or false positive firing. There needs to be some real intelligence behind this. “I am at home and have been here for 1–2 minutes. Please execute I AM HOME protocols.” vs “I drove past my house and unlocked all my doors.”

Better Proactive AI like Google Now (too much nagging? Too many notifications?) Not sure that we would accept our digital companion talking to us the way a real companion would. Would we find it annoying?

Cooperative Siri – I now have Siri on my iphone and my watch and my ipad. Is there any way for them to work together? Will they conspire against me now that they outnumber me? Do I compute better the more instances of Siri I have? Will that equate to having my own little Downton Abbey staff composed of little black rectangles? Imagine the next level of complexity that iOS can go to with a smarter AI coupled with a personal area network of cooperative devices. Getting all that into an Amazon Echo style unit, even if it is integrated into a new Apple TV, would be another level above the competition, and should link into HomeKit.

Siri integrated into apps such as iMessage where it recommends things into the conversation based on content from both sides but only you see.

Local iCloud backups with Time Machine. Time Machine no longer works the way it once did or even should, for many reasons. The personal backup problem has never been adequately solved: either the backups rely solely on the internet and are painfully slow, or rely on a local device like SpaceMonkey or Pogoplug or a USB drive that will disappear when your home burns down or shortly goes out of business. A device that works with Mac and iOS, and gives the ease and response of iCloud encrypted file access at local speeds with the ability to have files synced into the cloud would be a vast improvement. Get that Time Capsule functionality integrated with a HomeKit/Apple TV hub unit would be quite the killer unit.

Notice simple patterns in calendar events to predetermine frequency. Natural language calendar entry leads right into just having your digital assistant do it for you. If you can set a list of someday/maybe goals (in GTD speak) then implementing automated scheduling for Personal Goals is a brilliant idea as long as it includes location awareness as context. GTD is obviously more than just a wishlist of hobby tasks but introducing people to the basics is a good step for all. Speaking of getting things done, how about nested task lists?

3D Touch for speed dial. Remember speed dial? We may be better off with a menu that just brings up the list of favorites faces from iOS8 or better yet various groups.
In all, there are many reasons to love iOS and still many other things that can use some improvement. I hope to see what surprises WWDC 2016 brings us and can’t wait to be an iOS 10 user. Or will it be iOS X? Thank you for reading this series!

This post is the penultimate in my list of iOS 10 features I want to see. Seeing this series for the first time? Begin at the beginning. 

Attachments

I would like to see file attachments capability in other places than just the email App. For example, I want to send any kind of file I want in iMessage. Why? To take advantage of it’s advanced non-SSL encryption. Which you are at it, offer that encryption as an api to iOS developers to start securing more app communication.

App Store

Now that Apple is talking about Paid Search in the App Store, it seems like they are ready to work on more advanced features.

As much as Apple likes to brag about the number of apps that are available, at this point in the App Store’s history, there are too many apps that just don’t matter anymore. It would be great to filter apps by the last time they were updated in the App Store. Too many developers put an app out there and then abandon it before it is a year old, leading to the terrible clutter of the island of misfit unsupported and abandoned apps. Adding search improvements to filter by date to remove old irrelevant apps from searches can help users find relevant apps more quickly. A search filter for apps that were recently on your device or recently removed would be helpful for when iOS decides that it is going to half update an app and then forgets the app exists the next time you reboot – an unfortunately all too common thing these days. This is not the same thing as sorting by recently installed apps.

How about smarter personalized recommendations? I really don’t have games on my phone, so why do I keep getting recommendations for “the latest super amazing gem match puzzler” or “yet another zombie action game that will leave you occupied for hours” or “the buy our in-game gem purchases trope?” Frankly I think these games are ruined by these gimmicks and I’d rather not have anything to do with them.

It would also be cool if iOS could use Proactive to tell you about apps you have NOT used in some time, like maybe 6 months. I haven’t played Candy Crush in about a year but some weird nostalgia keeps me from removing it. Rather than fall to digital hoarding, I should just realize I need to delete it.

Lastly, let me try an app for just one day before I commit to buy it! There have been a couple of expensive and disappointing app purchases I’ve made lately and I would never have made those purchases had I been able to try out the app. I feel in some sense that this is done purposely to trap me into wasting money.

Game Center

Get rid of that darn Game Center banner that slides down!!! Who likes that thing?? I don’t play many games on my phone. Anything I have installed is usually productivity related so any games do not last long. Still that banner is highly annoying every time it interrupts.

(Continued in the final post, Part 11)

This post continues my list of  iOS 10 features I want to see. You can start the series here with Part 1 if you’ve missed the action so far. (Part 2, part 3, part 4, part 5, part 6, part 7, and part 8.)

Password Security

Given the FBI case early this year, Apple is looking to increase security overall on the device. One of the weakest points is the terrible passwords most people use. A dedicated password management and password generation app linked with iCloud Keychain and secured through TouchID is just what we need. Set a standard to work with 1Password and LastPass etc. It is entirely possible for my job to require that I use Keepass files for credential management and that I prefer 1Password for my personal credentials; giving Keychain a way to work with all of those would be much less confusing. Spotlight already allows deep access into all of those apps. Take it further to make things aggregated and simpler. Password Chef is a very original app idea, for instance, and something that makes perfect sense for getting the general public to improve the way they generate better passwords.

Multitasking

Side by side app viewing on Plus models – this is probably not going to happen any time soon, but given that Reachability allows you to see and work with a small screen interface, albeit briefly, there is no reason to argue against having the same small interface in a side by side wide screen view on the larger iPhone Plus models. Two iPhone SE screens side by side are about as wide as an iPhone 6S Plus is long. The reduced span in height is still usable work space and the two screens at once are now known to improve productivity.

Multiple timers

Why are we only allowed one timer? iOS should be able to manage more than one of these by now, even throug Siri, in combination with reminders. Uses: cooking, housework, repairs, lots of possibilities. Remember that the device is supposed to be our personal assistant. Remind me of multiple things I am currently actively juggling!

Control Center

With the advent of Night Shift, Apple has updated the Control Center across devices for the first time and brings up the idea that the user should also have a hand in choosing what controls are a slide away. Allowing users to select which icons are displayed and where they are displayed given their personal preference would be a great way of giving us choice. I would love to see 3D Touch take hold in Control Center: let me select a Wifi SSID using 3D Touch, for instance. Apple also has a lot of experience with complications from the Apple Watch so they have a better idea of what works and what doesn’t for swappable system icons. For instance, I never use the timer or the camera from control center but would love to put other functionality in its place.

Night Shift settings

Apple made a great change when adding the Night Shift feature. When using it I really do feel that it helps; placebo effect or not, it is still an effect. I feel however that more can be done with this compared to something like F.lux which I downloaded and compiled onto my devices in the short window before Apple forced it to be removed from github. Brightness needs to go down to the same dimness that you can get from the Zoom triple-click accessibility method. The color temperature needs to go even warmer to remove more blue. The effect is not warm enough for me when it is time to go to bed and I think the visual cue can be stronger. I’ve found that even when it is in effect, I still have to triple-click the home button when I wake up to a call from work in the middle of the night due to the brightness just so I can look directly at the screen.

Enjoying this series? Please share in the social media of your choice!

(Continued in part 10)

This post continues my list of  iOS 10 features I want to see. If you haven’t read from the start, begin here. (Part 2, part 3, part 4, part 5, part 6 and part 7.)

Document Management

For productivity users, handling docs from various sources is important. PDF is really the most ubiquitous of all the docs and a sort of cottage industry has evolved around the need for editing or marking these up or just plain document search. iBooks recently received an upgrade in 9.3 to allow PDF sync, but some of it does not work the way it does for EPub or Mobi. These improvements are very welcome and will lead me to shift away from Google Books right away, but iBooks needs some improvement before I find it reliable enough for all of my PDF use. I can’t help but think that iBooks needs some way to hook into the various other places I keep docs and that it can be a good place for keeping other personal documentation given a better ability to classify and sort the docs. Real document management would be something worth buying all in with iBooks.

iCloud

Storage PER DEVICE – if I own all these products, why wouldn’t Apple give a minimum amount of iCloud storage per device so that I can accumulate storage per active device? I’m probably going to buy iCloud storage anyway because I’ve found I really do want to store more photos and files, but the current 5GB limit per user is as laughable as a 16GB iOS device is for anyone who wants to store more than a handful of photos, songs, and games.

iOS needs to improve its sync and deletion for messages and notifications across devices. Why do I need to delete or read an iMessage on my iPad when I have already done so on my iPhone for that exact object? In the majority of cases (not all), when I mark something as read on my Apple Watch, it is also marked as such on my iPhone. Ditto for notifications status sync – if I read it on my Mac why is the notification still there on my watch or iPad? Something is being lost in translation between these devices and fixing this would mean less wasted time and confusion.

Better one-handed use

I knew going into using a 6/6S Plus device that parts of the interface would be out of reach of my right thumb, requiring me to use two hands to operate. While I do have a unique way of holding the phone one-handed, it is a bit precarious and has caused some scary falls. I’m sure this will result in heartbreak at some point no matter how much protection I have on my phone and I think Apple can make changes to improve that for everybody. Improving reachability range without using Reachability seems to be a challenge because Reachability itself seemed to be a stretch (no pun intended) of a solution. I have long since turned off Reachability because I constantly triggered it accidentally. I would instead prefer that screen elements be laid out more intelligently. There seems to be no good reasoning why some interfaces require buttons on the bottom and others require the same buttons at the top, but generally putting buttons at the top of the interface is a bad idea on the larger phones. It should be common custom that buttons which would be part of a flow should not require you to go back and forth across the interface if they are meant to be clicked or pressed in quick succession.

(Continued in part 9)

This post continues my list of  iOS 10 features I want to see. You can catch up with the first post here. (Part 2, part 3, part 4, part 5 and part 6.)

News

A real RSS based iCloud news service is needed, one that is baked into the OS to combine various services from around the interface such as Reading List and News and third party feeds such as Instant Articles. Reading List on its own is very buggy and doesnt let me make mass changes without immediately falling into the “syncing bookmarks” problem when I try to do two things in rapid succession making it pretty unusable. This is undoubtedly due to the fact that Reading List is tied to the Safari bookmarks which seems like a very poor choice. It is time to move it into iCloud Drive and stop the frustrating madness.

Aggregation is key to giving users a single interface to call home and dominate the attention. The river of news is still an important concept, and it is something you cannot get anymore from Facebook or Twitter because of how they no longer guarantee that posts will appear in order, just that they will appear in the most convenient fashion. Feedly, for example, keeps track of all the stories from feeds and doesn’t miss anything, even if I miss a day reading it. I can still go back and read past articles I might have missed from the day before. I find that a lot of news apps miss that point and only show you what is relevant at exactly that moment because they are actually catering to advertisers and eyeballs. I like keeping up to date – having one place to view it in makes life easier. There is still value in yesterday’s news even in politics and sports.

Music and audio

Music organization on the iPhone is very disorganized right now. The fact is that people try out and purchase content on different services and Apple should try to unify that with a common API. Spotlight search is a step in the right direction but there is more that can be done. Apple needs to stop pretending that its music app is the dominant one and that it shares the marketplace. While it is always shooting for knocking out the competition, they should at least not pretend they do not exist and instead try to make things easier for the customer. They should have a way of aggregating music search where owned music bubbles up above all services, or set a default/preferred (that’s why settings were called preferences, remember) music service that always shows above others. The reasons for having a preferred music service are varied and up to each person but those should be respected if the customer is really considered paramount.

Audio output on iOS in general needs improvement. The device already plays beeps and other sounds, while I listen to a podcast or music while playing a game. Take that further and allow apps like YouTube to also play in the background while audio plays from other sources. What we really want is multiple audio channels a la Audiobus and at this point we really just want Audiobus support!

I think that if the iPhone 7 gets rid of the audio jack it will be a major step backwards. I do not see the lightning connector as a good proxy for the headphone jack; lightning cables break way too often on me, I am now on my third pair of bluetooth headphones in one year, and I have yet to have a pair of EarPods go bad on me, no matter how badly I find them tangled and smooshed at the bottom of my bag.

The long standing complaint of not being able to save MP3s etc directly to the Music app makes it a bad choice for some people. There are people who have legitimate needs to store mp3 audio directly on the device without having to go through a third party app. I think it not only promotes the Apple Music store but can be a further source of boosted income. We know that many people that pirate music go on to buy the content after having tried it, but where they do so is up to them. If Apple could guide these users within their own app, it can guilt them into buying it from them. By using some sort of Shazam style capability in the background, the device could identify files that represent music that is also sold in the Apple Music store and unobtrusively display something like an iAd that directs the person to the Apple Music store to purchase the content. All this requires some sort of safari download manager, but how much horsepower and extra size would that really need? How much trouble would it be to allow those files to be used directly in various places around iOS without requiring ye olde thyme method of our grandfathers with laptop or desktop iTunes syncing. Why is iTunes even still a thing?

(To be continued in part 8)