Bose AR Unity Workshop | Highlights and Annotations by Gistr.

Bose AR Unity workshops teach building audio-augmented reality experiences. Learn to integrate Bose AR SDK in Unity, utilize device sensors (accelerometer, gyroscope, magnetometer), and create spatial audio applications for Bose Frames and headphones. Tutorials cover setup, connection, and 3D sound manipulation. We're here this weekend and we're doing a game jam with their new AR technology. We've given the developers a brand new version of the Bose AR SDK and Unity plugin. It allows Unity developers to easily add Rosear integration to community-based apps. They've built a very intuitive system for our developers like me to be able to just pop an immunity and just go, we give you access to all of the sensors that are available in the device, The accelerometer, the gyroscope, two different rotation sensors, they provide orientation of the user's head and world space. We can actually provide spatialized audio which just gives you an entirely new level of context when you're making any sort of experience. We're able to get all sorts of different types of information from the frames glasses which we can use to help develop new types of interactions. We were challenged to create an augmented reality experience that put audio first. It's very interesting to look at games from like a whole different angle. Usually people think AR they go it's all visual and this one. it's both so it's audio. The challenge really made us put our heads together and be like what can you really do? There's a whole world of sound and we shouldn't just be paying attention to just the visual components so we've had a lot of really creative ideas that the developers have come up with so far and I think some of the This segment showcases several diverse games developed during the game jam, each emphasizing audio interaction. Examples include a game where players dodge unseen bullets based on sound, a catapult game relying on directional audio cues, and a spy thriller using radio chatter for navigation. The speaker highlights the potential of audio in AR and the innovative ways developers are using it to create immersive experiences. uh is also like a really cool example so i'll just go let him explain it that is about popping and grouping your head to the music similar to other rhythm games, you score points by staying on the beat. At the moment, we have about a dozen custom written songs in the future. We want the possibility for players to download hundreds of songs or play with their own music library. [MUSIC] We're going to play party. The washing machine, our foxy friend here is grooving along as well. You could follow it and he helped you stay on the beat. The line below is showing your head movements. It also changes colors to indicate how well you're doing. We really intend this game to be a phone and pocket experience. We added both audio and tactile feedback. For example, the high pass filter turns on when you're doing bad and your phone vibrates with the beat when you're doing great. So the idea of hip-hop hero came from us wanted to make a game that helped naturally play with the bose ar devices something you can play while competing on the train in public what else do I have here? Um, yes. so I have a few, um, actual like gameplay footage that i'd like to show you on some experiences that I personally like. Um, developers cut up these videos to kind of show what what the games are like. So just gonna get into these again, just some, some more inspiration. The worst. Grim Reaper soul mates is an AR music t-shirt in which a singing Grim reaper plays a song that responds to you and your world. Grim Reaper Sebastian is a character from another one of our games. He's tired of his dead-end job. collecting souls and wants to pursue his dream of being a songwriter seeking inspiration. Sebastian asks you to let him write a song about your life [music] You were looking out maybe lost [music] there's magic in almost every place [music] would you like to share your thoughts with me? [music] wanting to learn about the inside of the human skull? I ask what's going on in your soul? This segment showcases "Flower Garden," an AR game where players explore a world of musical flowers, collecting song loops and layering them to create unique musical compositions. The speaker emphasizes the game's intuitive design and the clever algorithm that ensures harmonious sound combinations, even with complex layering. The segment highlights the potential of AR for creative music experiences.This segment provides information about Bose's AR certification program, which assists developers in publishing their AR apps and potentially gaining visibility through features on Bose's platforms. The speaker also mentions the developer portal and its resources, including case studies and SDK downloads, showing the support Bose offers to its AR developers. once you're all imported you'll see these three folders show up so the first one that i want you to look at is the bose folder if you go inside the bose folder there's wearable and then inside wearable there's modules you go to modules and the first module that we want is connection so we're going to go to connection and then prefabs all right so that's bose wearable modules connection prefabs and i want you to pull in this first prefab that's available here uh so just drag that into your scene and you should see this this white rectangle pop up. Now the first thing that I like to do, um, when I start like a unity project, especially if I'm developing on mobile, is I like to pull my game view, which is usually up here. You might have a different layout and I actually like to take the game view and I like to make it visible. So I actually drag that game view over here to the right side of the project panel. So I always kind of have a little preview window here of, of what the what my preview looks like basically. So if you go ahead and do that, then you'll see that the panel that we just pulled in actually shows up here. So and I'll explain what the wearable connect panel does. It's basically the entire menu that shows up that that you can browse all available devices, you click a device, and you connect to it. Um, so this handles all of the, the connection for you. There's some cool features on here, like auto reconnect and and such. So for example, if during your app for exam, maybe Bluetooth connection turns off or or for whatever reason the battery runs out, this panel will pop up again. it'll say you've been disconnected, you know, searching for a device to kind of like keep the user experience clean. So you have the wearable connect UI panel. And the next thing that we actually need to make the wearable connect UI panel work is we need an event system. So what I want you to do is I actually want you to just create an empty game object somewhere in the scene and call it event system. Okay, and on that empty game object, I'm going to add component here in the inspector. And i'm just going to start typing event system. There it is. It's a built-in unity thing. And once you have your event system, there's a button there called add default input modules. So just click that. Now explain what this does. This actually allows you to click the UI of the connect panel. Now, the reason why we do this separate is maybe you have an already existing unity project. and maybe you already have your own event system that you've set up. So we don't want to mess with that. But since we're starting a new project, we're going to create this event system and add the default input modules. Like I said, that just allows you to actually click the UI and and select the panel. So that's pretty much it, the last piece that we need to actually, um, set up like our entire Bose ar is we actually go in and we're going to create another empty object and we're going to call this one wearable control. Now what this will do this will actually stream the sensor data back and forth from our app. So wearable control And I'm going to add component. Now if you clear the search here i'm just going to show you where the Bose components are. So when you clear the search there is a Bose subcategory here and you can click bose wearable and we have some sample scripts here that you can use. So the one that we want is wearable control. We're just going to click that. Now one other useful thing that we added semi-recently is uh, you can actually go up into let's see here. it's yeah. so if you go to asset for assets. create and then there's a bose wearable category and there's an app intent profile so i'm actually going to take that app intent profile i'm going to put that like in my top directory and and assets because I was in the in the prefabs folder here you don't have to but it's just cleaner that way and what you'll see here on the right is you'll actually see the sensors that are available to us so for this first app that we're building um we just need the gyroscope so i'm gonna check the gyroscope on here and rotation sixth off and then the update interval i'm gonna make it uh let's say 40 milliseconds that's all we kind of need in terms of sensors and what this basically does is it just kind of syncs up what what the app requirements are and what what the app is going to be using so back to our wearable control that we created earlier. you'll see there's a active app intent profile Empty slot right here. You can actually just click that and just pull in that app intent profile that we just created So again, that's uh, app intent profile is in assets create bose wearable app intent profile in the app intent profile You want to select gyroscope rotation sixth off and 40 milliseconds And then in wearable control just you know Put that app intent profile there. The last step to get this to actually work is where it says here you see it says editor default provider It says debug provider We're actually going to switch that to usb provider And what that will actually allow us to do is we'll actually allow us to send the data over usb bridge directly To Unity Which is great for debugging. Um, while you're developing so that's actually what we're going to be doing first so your wearable control has a usb provider and that's pretty much it so your app will actually now connect both using usb and if you built this to android or ios you'll actually be able to pair with bluetooth the only problem is is we're not getting any of the sensor data so we're not actually doing anything with it yet so the first thing that I want to do is let's say I just want to uh visualize like the head orientation like on the screen so how do we do that right so what we're going to do here is I actually want you to go back to the modules and in here there's another folder called model loader and prefabs so if you go to bose wearable modules model loader prefabs in here you'll actually see we have a 3d model of all of our Bose AR enabled wearables which is actually really cool and then there's also a default for if it doesn't know or maybe for a future product or something like that so the wearable model loader actually pulls up the appropriate model automatically so it knows which one you're doing but right now let's just do it manually so pull up the one that you have so I have Rondos here so i'm just going to take the rondos prefab I'm just going to drag it into my scene and if I double click it you'll see it's very small so I would recommend to scale it up let's say to something like 40 by 40 by 40. so we just have our roundels like that in the scene and the last thing that I want to do is on this prefab i actually want to add component and in here we're going to go to bose wearable and then there's a script called rotation matcher so we're going to select that and we're going to make the update interval 40 milliseconds okay so what rotation matcher does is what the name implies it'll match the rotation of the wearable so it'll actually take the gyroscope data um and that's actually pretty much it so this should already work so what we're going to do is we're actually going to test it out so I want you to do is I want you to take your frames hook them up with usb to your computer if you're on windows you'll hear like the connected sound and I'm actually just going to maximize on play here so you can see what I'm doing I'm going to press play it's the picker is there I have my frames. I select it in the UI it's going to connect to it and there are my frames tracking the rotation as intended. This is all happening over USB so the cable disconnects. yep it will stop. So the idea is that this is for for development. You're obviously going to be using the cable but if you took this app and actually built it to your phone it would work over Bluetooth. So I want everyone to just get to this point and then we'll move on to the next part. And This segment addresses common connection problems users might encounter, such as failed connections or lack of sensor data. It provides practical troubleshooting steps, including checking for firmware updates using the Bose Updater tool and verifying the proper configuration of the Wearable Control script and event system. It also includes tips on how to turn off the Bose Frames. This segment shows how to spatialize audio using the Bose Frames as the audio output device. It explains how to route audio through the Frames on both Mac and Windows systems and demonstrates a simple example of placing a 3D sound source in the scene. The segment also covers the use of Unity's built-in spatial audio capabilities, including the adjustment of parameters like spatial blend and Doppler effect.