navarre.aiClarisTalk AI · the FileMaker AI podcast
Summary of WWDC and Apple's AI strategy, DotFMP Berlin, The Claristotle learning initiative
Show Notes: Introducing Apple’s on device Foundation models: https://machinelearning.apple.com/research/introducing-apple-foundation-models Intuition from iSolutionsAI: http://isolutionsai.com/intuition/ Links: Claris Community Live: Learn about AI prompting basics: https://www.youtube.com/watch?v=AQlOu-BIs5U See Cris at Summerfest-Tech “A.I Deployed: Real World Examples” on WED June 26th in Milwa
Apple PodcastsSpotifyYouTubeAll episodes
Transcript
Transcribed by machine (whisper, large model), 14,216 words. Names and product terms are sometimes misheard; the audio is the record.
Claris Talk AI. Welcome to Claris Talk AI. I am Matt Navarro. And I'm Chris Zipolite. Hey, Matt. Good to see you. Today, we have some good stuff on our agenda. So the Apple www.wwdc was last week, I guess. I was at the .fm conference just before that in Berlin. We want to talk about Claris Dottel. Yeah, exciting. I know. Yeah, that's the learning initiative. And you've got something coming out, don't you? I have something. Well, it's something in development. I would say pretty early development. And we'll talk about that. And then you have something to talk about as well, which will be involved in some screen sharing. So people, that's what's coming up. Yes, sir.
Or person, I don't know. I'm not sure. I think it might be person. Moms. Yeah, at least we have two people listening. But yeah, this will be a good one to check out. She's watching from that. Oh, man. Well, I. I hope there's good reception. Oh, my goodness. Well, Starlink actually reaches out there, actually. So, yeah. Let's just move on. Please. So, WWDC, what's your take? Well, so I'll admit. Well, you know, we, you and I talked a ton about it. Sure. You know, absolutely. Yeah. And, you know, I've been paying a lot of attention to what I thought was going to happen. And I. I found myself about 50 minutes into the. Now, just to be clear that the presentation that I'm alluding to right now was the 10 a.m.
Pacific one that, you know, was there pre-recorded. Just looks like every other, you know, product launch ever. And I remind I had to remind myself and I think we should all remind ourselves that Apple is essentially a B2C company. They're a consumer. They cater to consumers. Like when we watch Google, Google's announcements or Microsoft's announcements are opening. Like, yeah, they're catering to businesses. They're trying to find business use cases. Right. I had to remind myself 50 minutes in that, like, you know, then we're talking about emojis and. Yeah. Just silly things. Yeah. That that actually is their market. And so I if I was being totally honest with you, I was taking notes and I was saying to myself, like, hmm, did they miss again?
Right. That was my initial thought. What was yours originally? Because that's just a tease. Well, I guess I kind of knew about that part. But what really surprised me is that they have the fastest iPad they've ever made shipping now. It's always a part of those things. The fastest. I frankly missed it completely. But just always, you know, of course, the new version is always faster than the old one. But anyway. Well, I think the thinnest, the thinnest. Anyway, it doesn't matter. So, yeah, the first part was like hardware, software, you know, stuff. And then the second part. Second part. Yeah. Well, so then after the then after the stock essentially dropped almost three percent overnight and everyone was completely underwhelmed by what they had seen.
My notes for our discussion essentially said and I'll admit I put something on our Slack channel that we used to coordinate this that essentially said there's nothing to talk about. Let's find something else. I quickly edited that the next day because the the the presentation that people should, in fact, check out, it was the State of the Union. And that's when they talked about how all this is actually working. So we're referring to the Apple intelligence and there's a lot to unpack here. But I think if we focus first on probably the most profound part of it was, well, first of all, let me just spoil it. The big thing that I think made the stock drop and then, of course, it skyrocketed the next day.
So congrats for everybody that bought the dip. But the thing that underwhelmed everybody was that they said, oh, the Apple doesn't have. They don't have their own models, right? Like this was their big chance to say, well, yeah, we've got one, too. They do. They actually have two, two of their own proprietary foundation models. And we'll go into that here in just a moment. But that's a bit of a spoiler. Like people just thought, oh, well, they did a, you know, a deal with OpenAI. And they have a deal with OpenAI, right? They absolutely do. I really like that, that the description of that, which we'll talk about. But I think I don't know. So we made predictions.
I think we did pretty good. Yes. Oh, no doubt. I mean, like. They hit privacy. They hit that privacy nail extremely hard. Nailed it. So, yeah, if we, if we, let's take a look at the personal context part of it. That was the private. That's really where the privacy comes into play. So the way we had framed it up, you know, just to kind of set the stage for this is that we said, what are they going to do? How are they going to train a model? And by the way, first of all, this is kind of a big deal. So we were saying, how are they going to train a model? Because they have this position on data privacy. Yep. So then if you have a position on data privacy, how do you train a model?
Well, here is the answer. I dug very deep into this and I finally found the answer. So here's their official statement on the data they use to train their two foundation models, which we'll go into it here in a second. Training data includes licensed data and publicly available data collected by Applebot. Applebot. Apple has a has a web crawler. Did you know this? I did not know this. So. I did a little bit of research and sure enough, they do have they have a web crawler. It's on the Apple website. And it's not even new. This is apparently something that they've had for a long time. So their statement on it is it's a licensed data and publicly available data collected by Applebot.
Apple's web crawler web publishers can, in fact, opt out of having their content used for Apple intelligence training. Yep. That would probably be something good to know before the training. But, you know, you can go in there and opt out if you don't want to be part of future training. Right. So so they had they basically had to train models. And so they did it with their own data. And then I'm sure to some degree they I didn't hear any. Maybe I've missed some announcements on the on the licensing agreements that they came up with. But they came up with two different models to do that. So that's kind of a big deal. Now, we said, great. That's one of the privacy things.
The second thing is what data? Does Apple have access to that? Nobody has access to. And you and I talked a lot about their wall. Yeah, me, the walled garden. And and so to me, that is hugely profound. Like, I don't know if there's many organizations, at least at the scale that Apple can, that can do the type of things that we're going to see with personal intelligence. I mean, our personal context, personal. If you're fully in the Google ecosystem, wouldn't they have a huge amount of information on you? I would have fully. Fully expect. Well, actually, this is an interesting point. Google for workplace and Microsoft, whatever the work thing is, we're like here at iSolutions.
We're a Google workplace. And so the Microsoft 365, the copilot concept. Right. So what's actually happening in those two cases? Microsoft specifically has something called the Microsoft graph, which is really what copilot is for for work is. You've got all your stuff on their platform, meaning you've got all your PowerPoint presentations and your Google Docs or wait, I'm sorry, whatever the Microsoft ones are. And so it has the real technologies. It creates this graph database, which is basically a data lake of all your all your organization's data. And then what copilot does is it allows users to interact with that. So when you're buying copilot, you know, the revolutionary thing there isn't the copilot part of it.
It's the fact that all your data is pulled together in one place and now you can interact. Now, it's kind of a nothing burger if you don't have any of your information on the Microsoft platform or the Google workspaces platform and Google workspaces formally just announced there. I mean, I think they've kind of had it for a while, but they formally just sent an email out like two days ago that says, hey, all your Google Docs and Google Sheets and your Gmail and all that stuff is now, you know, accessible. So that's not revolutionary. But what Apple did is they said, hey, Matt, all your text messages. And emails and various interactions that you have on device within the private wall garden that we all trust are now available to what I'm referring to as your personal graph.
So that's what they're what they're talking about when they talk about personal context. And I think a couple of big things. Right. So there's that the data never goes to Apple servers. And the AI processing actually happens on the processor on my device, not in the cloud. And unless that there's something where it does. Yeah. Those are that's really, really big. And I think that's also kind of what we predicted because of the privacy stance. Well, and also the some acquisitions that they made were all about running models on the device. Well, yeah, we did get a clue. So there were definitely some breadcrumbs now. So what you just mentioned is a really big deal.
So so that's how they're facilitating that. And I want to show I'm going to share. I'll have the YouTube folks get ready in a second. So I'm going to share what we're talking about and show show this what they released. But to speak to the personal context thing. This is such an important idea when it comes to interacting with language models in general. Let me let me zoom out for a second and then we'll get back in zooming out. Like I did a prompting webinar, a prompting basics webinar. And one of the principles that I shared with everybody is instead of saying, you know, when you're prompting and you like, I think everybody when they first start prompting or they start looking up prompting guidelines.
They run across, say, to the model, you are a helpful assistant, right? Like, like, like somehow it's not helpful or it's not an assistant. Like, it's not about when you're prompting and you're interacting with these language models. It's a nothing burger to tell it what it is. You tell it what who you are is the point. So we have this concept of called user personas. And every one of our I talked about this in the Gange conference and some of the user personas that we write for our deployments. Our pages and pages long and like, and then in addition to the user persona, like on our intuition platform, which I'll show a little bit later on. We can identify these are multi-user platforms.
So we identify who each user is. Each user gets their own persona, personal intelligence, right? They are personal. I keep calling it personal intelligence. It's a personal context. And why is personal context meaningful? Because you care about certain things. Like if you and I work in the same company. You care about certain things to do your job. I care about certain things to do my job. I look at information a certain way, or I have access to certain information, whatever it is. So you and I could ask the same exact question to the same exact language model and get significantly different answers because if it knows who I am versus who you are. So now source of truth thing that I let you talk about, right?
All, all the time. So, so providing the truth and then also telling it who I am and what I care about, that's how we actually focus the aperture on, on what I care about. So that's, I would say if it's somewhat controversial, well, because none of these people are listening to us right now, but if for some reason you're just dismissing AI and you, cause like you asked a couple of questions to chat, it got the answers wrong. I am here to say that's just as much on you person, not listening right now as it is on the models. Like you have to know how to ask the questions. And so providing context about who you are helps you get those right. Now that's what Apple did here.
So it knows. I'm a little worried about maybe, well, I mean, I'm not, I'm not always being professional when I'm texting my friends and, you know, emailing people and stuff like that. So I wonder how much of that it's going to bring into my context, but I think there's really interesting intelligence behind all that. So your thoughts on your thoughts on how do you feel about letting Apple into your walled garden to make your, your AI experience more, more personalized? I actually think that the. I actually think that the amount of data that my Apple watch knows about me is insane. If you trust that, like it literally knows how your heart is pumping, right?
Like, like it knows more about you than you do, frankly. Right. So I think we've already gone to the level of extending trust. I mean, I doubt they do it, but if I'm watching a movie on Apple TV, it can monitor my heart rate and see if, and see if I like the movie or not. Right. It can presume that. Oh, I love that. That's actually a brilliant idea. Um, yeah. And I mean, algorithms have been. They've been tracking our likes and dislikes forever. Like every time you go onto Netflix and say, you don't like, by the way, here's a little secret. Every time I go and I don't like a movie on Netflix, I go pick another movie somewhere and don't like it as well, just to balance my algorithm out.
So I don't want it to think like I have like this bias or something like that. So I'm, I'm Apple has been, Apple knows all those decisions, uh, for us as well. So, okay. So let's talk about, let's talk about the models and the performance of them and kind of where it's going to get done. So first, if anyone's wondering what the ultimate resource is, this is going to be in the show notes. This is the paper, uh, called introducing Apple's on device and server foundation models. They don't have like a fancy name for them, like, you know, Gemini or something like that. They're just Apple's on device and server foundation models. So, um, oh, you know, when you get past the hypnotic header here, um, this was all the stuff that was reviewed essentially in the, and there are some sessions, but mostly in the state of the union.
So first off, let's take a look at this part. Um, we talked a little bit about the training. Let's actually go. Yeah. We're going to look at the architecture here. So they did something really interesting when they trained these models for starters, they did this concept of adapters, which is, I'm still trying to wrap my head around it, but it seems very similar to me to like this mix of experts approach. If anyone listening is familiar with that, it really just sort of like segmented out. Um, you'll, and for those that are looking on screen, it segments out summarization, proofreading. mail replies, that kind of stuff. So it's really saying like, Hey, um, this part of the, the architecture is assigned for summarization.
So you go do summarization and you do tone adjustment and all that. And then they all kind of come together, um, you know, as one now. So multimodal, you mean kind of a thing. Yeah. Um, yeah. Multimodal generally indicates like its ability to process images or video or audio and types of things, not just different, different models for text processing. Like we just saw. Yeah. Like different. Different expertise, basically. Right. So, um, let's, let's look at this. So here's actually, this is what the architecture really looks like. So, um, so down here, so to, to kind of talk people through it, the Silicon is such an important part of this, right? Like, like they've been moving in this direction and it is meaningful to us.
And by the way, I just like ordered a M three max to, to get ready for all this, the, the super light and thin, um, iPad that you mentioned in. Just earlier actually is intended to be able to run these types of things. Um, and shoot, you know, now's as good a time as any to just mention this, but for those that are interested in this stuff, I did a little sleuthing and found out that in order to run all these things, whenever the heck they come out, not every phone or device is going to be able to run them. So, uh, we're showing the list on screen, but you basically, the, the, all the way back to the iPhone XR and 12, dang, or 11. Isn't that crazy?
Yeah. iPhone 11. So basically, so like my wife has a iPhone 13 mini, cause she loves the mini cause it fits in her purses and stuff. And I was, I looked this up to see if she'd be able to run the AI stuff. So yeah, all the way back to XR. So basically anything that can run, uh, iOS 17 can run these. And that was kind of interesting because I thought it was going to be exclusive to the Apple Silicon devices. Right. So you'll notice that they're leveraging their Silicon both on device, which is for those viewing, that's what's on the left. And, and on the servers as well to their servers, right. Also Apple Silicon. That was fascinating. And I don't, I mean, we don't have access to these, um, servers ourselves, but over here on the upper right hand corner, and sorry for those that are listening, I'm just pointing to the server stack.
This is the thing that we just looked at a little bit, you know, that has these, uh, adapters in there. Uh, you can kind of see visually Matt that they've got adapters going in the language and the image side. So without getting too into the weeds, this adapter. This adapter thing is kind of one of the things that's actually makes them special. So over here on device and left-hand side, but, uh, above the Silicon is where the on device models actually live, which is really, really cool. And it has a semantic index, which is, uh, akin to, you know, we've talked a lot about vector databases and searching and the vector index it's basically on device, semantic vector database for practical purposes.
And then the app intense toolbox. We're going to get to that in a second. That is actually kind of a big deal as it pertains to us, Claire's folks. And then up on the top are all the apps in the experiences, Siri, the writing tools, your favorite new tool, the Jumoji, uh, and image playground, which I know you can't wait to get your hands. I entirely forgot about that whole and Jumoji thing. You keep bringing it up, but whatever. Oh, cause I, I I'm so excited about it. The Memojis totally missed me too. I whatever. Um, okay. Now this, this chart here, you're looking at the writing benchmarks on device server above. Totally confused me because they're talking about these models and, um, right.
So Apple on device, like on your device, mystery seven B Gemma, seven B B dash three. Many. So first of all, what, what do we, why are we even looking at writing benchmarks? Well, first of all, they really announced three things that you can do on device slash push up to the server every now and then it's writing tools, which is like Grammarly and, you know, chat GPT or whatever, uh, image generation. Which is the, I won't mention it again, Jumoji and image playground. Um, and then the, the third one is, is, um, uh, well, it, uh, and I completely forgot what the third one is, but we'll get to that in a second, but here's the deal. What are benchmarks benchmarks generally, when a new model comes out, you go to hugging face and they've got all these different like leaderboards, they call them.
Um, and they list, they, they, they kind of agree. Like the industry agrees as a whole. Like, here's all the different ways that we can evaluate a model and they run it through these tests. And so every time a new model comes out, they, the, generally the model trainers will run the, um, them through the benchmarks. So we can compare. Apple did not use the benchmarks that the rest of the industry standard uses. They created their own benchmarks and instead of revealing how, you know, what the tests were, they basically said, here's how we stack up against other models now. Um, uh, so we're looking on the left-hand side. It on device and on the right-hand side as the server.
So here's how to interpret what this means. We're looking at the idea of summarization of tax, which is like when they talked about their feature of, which is a great use of, of language models, by the way, even in your applications, summarizing meetings or, um, yeah. I mean, like if you're not summarizing your meetings with AI, your, I, what are you doing? Like, what are you waiting for? Like, it just makes no sense not to do that anymore. Composition though, is where you're actually writing things, you know, like write me a blog article or something. So we see Apple on device model right on the top. Like they're, they're saying, Hey, these are the best models.
Well, let me give you some context here. If we just look at summarization, Apple on device is the top of the leaderboard, but it is only on top of, uh, these models called Mr. All seven B Gemma, seven B five, three mini and Gemma two B. What are those things? Those are really small seven B and two B and three B mean billion parameters just for some context. Uh, uh, open AI, uh, 3.5 GPT 3.5 was about 480 billion parameters. And these are 7 billion, 3 billion. Okay. Yeah, exactly. And so these, and so this is why this is super important. Like if you're thinking of, like, I talked a lot about engage about how important open source models are, for example, right?
The problem with open source models is they, they don't have the same capabilities of the foundation models up in the cloud. And so. And then also, if you're going to run them locally, you need a massive amount of compute, whereas you don't have to pay for token costs. But the amount of compute that you need is, is ridiculous to the point where my team, we basically have a threshold at about 10 billion parameters that you just can't run these in production at any level of cost. That makes any sense, right? Yeah. It's so, I guess I, I guess the question I would have for you, I think I had it a while ago. Um, how much compute does a, does a base. A basic query take on say chat GPT for, um, Omega Omicron for, oh, yeah, it's so, well, we don't know because you can't run those locally and, and, uh, open AI or slash or Microsoft for sure.
It takes that. So, so a good way to measure that is exactly what, you know, and I, I appreciate you bringing it up in this context. It's in the billion parameters, right? So I would say that about 10 times the size of the models that are running on device is about the maximum that you. Would reasonably like want to pay for, to run on like an AWS extra large. Um, yeah, I guess we get into like GPU available. I mean, like if I had, if I had a bunch of, let's just say I had a bunch of Mac minis or whatever, um, and I wanted to wire them up with a, with a model for, um, that was GTB for, oh, and I did a query and it came back with a response in two seconds.
How many Mac minis would it take? To produce the same result that I'm getting. It's kind of my question. Uh, what's the, yeah, literally we don't know the answer because that information is obfuscated from us because only opening I knows that answer. So the only way we would know that answer is if we use an open source model. And so that's what I mean by saying around like a, like a, like when you get up into the seven DB, you just can't, you can't reasonably run those on, on any kind of arc. First of all, Mac. So we're talking about hundreds of Mac minis then for some. Yeah, yeah. We're talking about actually. Not even, uh, CPUs, but GPUs. That's where this whole GPU conversation goes in play.
And yeah, you want to run on the Nvidia H one hundreds and, and, and that's why there's a space race for all that is because, you know, open AI needs like tens of thousands of those. Okay. So that's the server stuff. So the on device, obviously it's not going to be able to handle the same kind of a model. Well, so, so w w w why this is meaningful is because imagine a slider, Matt, where capabilities of the model and the compute necessary. Kind of like slide in this continuum. So you want to find the sweet spot. If you want to find a model that runs on your phone, whether it's an M three Apple Silicon or whatever, you can, you're moving that slider pretty far to the left on capabilities.
So we see evidence of this on this chart right here, where it says basically the Apple on device, when it comes to summarization and composition of text is akin to Mistral, which is a open, a French open source model, uh, seven B. Or Gemma, which is Gemini's, uh, and five three is Microsoft. Uh, seven B. I personally thought they were probably closer to the three B model. So I was kind of impressed that they were more than that. But the point is like, they're nowhere close to GPT 3.0, let alone 3.5 or four. Okay. Just so we're clear on that. So, but it doesn't need to be right. Cause what is it doing? Summarizing text and, and, and writing text. Cool. And again, remember we, we don't use models to answer questions.
That's not business use case. And that's silly talk. Yeah. So let's look at the servers. Server on the other hand. So, so just to kind of frame this up, why is this important? Well, you alluded to this earlier that sometimes somewhere in their architecture, they recognize this thing that Matt just asked Siri to do or something like that. Right. Right. It's going to re need a model. It's not even talking about compute. It's saying the model Mistral seven B or the seven B models aren't going to be enough for what you want to do. Right. So we want to kick it off to the server. Right. So then there's all this sophisticated stuff they do where they, they're, I don't know.
They didn't even really reveal it, but it's super private and you're taking your data and it's sending it up to the server to just perform this one task. And just to be really clear, this is not the role of the chat GPT plays. This is not sending it to the open AI models. I actually left the first presentation thinking that might be the case. So in that case, It is more akin to the performance of, as, as those who can see on screen GPT four turbo, which is not the four. Oh, it's the predecessor to that. Mixed draw mix of experts, which is eight by 22 B that's that to give you an idea, that's about the maximum mixed draw that one, the 22 billion parameter, that one's about what you can probably run on like an AWS Excel with some GPU access to give you an idea.
Okay. So but even on, so that's summarization of text and then composition, it, Apple says, well, we can't quite beat GPT four turbo at this point. We're talking about 9.5 versus 9.7. Those numbers are so close to each other, but those are just whatever Apple came up with for their metrics. So it's more about the ranking, like where they're close to than, than anything else. Right. So I guess my, here's my big question on this Apple server. So Apple has server farms with some kind of a computer that you can't buy that has Apple chips in it. CPU and GPU chips. Yeah. Uh, uh, yes, basically. Yeah. The server farms that Apple has been using all along or whatever.
Maybe have not been using the same kind of machines or the better, but other server farms are using, I guess I should, I don't know. Maybe that's something that other people know, but I didn't, I didn't know that. I, I don't think this, I'm not even entirely sure this is public knowledge prior to their review. Yeah. This announcement. I don't, I don't have any evidence of that at all. Um, uh, and by the way, the last thing I'll show you on comparing is just, um, they did another thing. I don't know if you guys are familiar with this, but they're basically saying win, lose, or tie, which is a unique way to benchmark this. Also super confusing, but yeah, go ahead.
So they're basically just saying, again, these are the way to look at a chart like this is just to simply say what, how these models perform closer to what models we can get our hands on and test already. And that's the Mistral 7Bs and the 3.5 Turbos and stuff like that. So, so I would say I'm not like dejected in any way by these being substandard to the, the frontier model. I think all of that shows that they're right on par, basically. They're on par. Yeah. They're, I mean, here's, here's, here's, I'll, I'll put a bow on that. You just don't need, you don't need a GPT-4 Omni for everything you do. I do. Okay. Why else would I, I mean, what else, you know?
Well, if you wanted to do. I don't switch to some other model. Well, understood. But I mean, like, if you're just saying, um, summarize these meeting notes, you can do that with GPT-4. And get them much, much faster. Yeah. Well, yeah. And cheaper as well, too. Right. So these are all factors in there. So here's, I, so let me, let's talk about some stuff that like, I dug a little deep. So my first thought was like, oh, cool. How do I get access to these models? Right. Like, is there an API? Cause like all the other, you know, open AI announces a new one. And later that day, you just go to their documentation, API documentation and figure out how to update it. Which by the way.
Um, allows us to swap out new model versions on announcements within like, within an hour. Like the, the, when GPT-4 Turbo went to GPT-4 Omni, we updated our intuition platform within 45 minutes to the new model. It's just APIs. Yeah. So swapping out to a new model is not a thing. Not any kind of a thing. Backwards compatibility. Not at all. As a matter of fact, when 3.5 or when 3 went to 3.5 or no, when 3.5 went to GPT-4. I was on an airplane flying home and I was like, oh my God, I want to record a video of this. When I get home, I got home and within an hour, I'd updated all my testing things to the new one. And, and actually, no, that was three point.
It basically. But they didn't deprecate the old one, right? The old ones stick around for a good long time. No, I did a, uh, I did a study and the least amount of time they gave you for deprecation between announcement to deprecation was six months. And to give you an idea, um, that's how, when, uh, QuickBooks online. Did a deprecation of some payment, uh, API that they did, they gave six months as well, to some, some APIs give like 12 months, but the minimum I found as a comparison was six months and open AI has never given you less time than that. The big one though, that I think people point to is, and actually that in the anecdote where I was flying home and I wanted to test was actually when the, um, the completion.
Endpoint change to the chat endpoint. So it really was kind of a fundamental change in the structure of your APIs, but even that one, like literally I got home. Sat here in less than an hour, extremely good news for, for people doing this. So, so tell me, I'm not sure what you were in the middle of, but I want to kind of finish up the, what takeaways you had from the Apple announcements. Okay. What's your, my, my takeaways were there's a gap in the, and then, and then we'll talk about how I think us Claire's folks can get our mitts on these things and what that means to us. So my takeaways though, were this. There's a huge. Void in AI for personal AI, a literal personal assistant.
And I'm not talking so much about the fact that we can talk to Siri and all that stuff that that's, that's the interface for this. Right. Um, I, I would say that Apple nailed it because they filled the void of my personal information. Uh, and I get to use that from an AI standpoint, as a matter of fact, um, I think this is the role that Siri was always meant to play. Um, you know, They, they, they, they, oh, stop. I'm not talking to you, Siri. I got a lot of three Siris now responding to me. We're not quite there yet, but, but the point being that, so we got Siri and we could ask you questions. And then for a year, we thought that was the coolest thing ever.
And that's what AI meant to most of us. That was like 12 years ago, by the way, just for some perspective. Yeah. And then like two years later, just all of us turned it off and none of it, we were so frustrated by its uselessness. Like, you know, giving me a list of web searches, like, come on. Like, it just didn't make any sense, but, but here's, here's, and maybe this can teach. So now it knows everything about me. As a matter of fact, um, ever since the announcement, I've just been logging all these examples of when hopefully the Siri with a personal Intel, uh, personal context, the Apple intelligence version will be more useful. Like last night I was with my wife and we were walking out of, um, we were walking out of a baseball game and we were like, oh, we should call someone.
So to do the blah, blah, blah. And I was like, I don't remember when she said she was coming back from her vacation. Like I should absolutely be able to say, Siri, just go. Was there the email or text or whatever? Go find out when so-and-so is going to come back and then whatever. Right. That is what we're going to be able to do. And that I think is a void that they filled. Did they say from a business computing standpoint, like look out open AI and Gemini, like here comes Apple. No, none of that happens. And nor does it, did it have to. I don't think it had to happen for, for us. To, to care about that. So, right. So great. I don't know. What was your reaction by the way?
Uh, so I mean, I, I was kind of impressed by a lot of this stuff for sure. I mean, I thought that, Hey, they've got it. They're on the right track. These, the single example they gave for Siri was so stupid. It was like, come on, man. The example was the flight thing. Give me directions to, no, it was like, give me directions to, uh, you know, Smith rock state park or whatever. Oh wait, no, I mean, Smith Creek state park. Oh, that was so dumb. Yeah. I'm sorry. That one was like, I was like, oh no, you had me for a second. And now I'm like, oh no, that's even worse. So, I mean, all, and by the way, that was a semantic use case, like, and I think if I bring that down to Claire's people level, one of the biggest frustrations we've had with, um, lexical searching was that like the John Jonathan thing, you know, or the DR period versus doctor thing.
Well, semantic searching semantics can help us with that. Right. And that's a tease for something that's coming up. Yeah. Um, so here's what I think just from a consumer Apple consumer, you know, fan boy standpoint, what this means. And then we'll talk about the Claire's thing and then we can get into that very tease that you just met. Um, here's where it's a big deal is not only does Apple have all your personal context and does it have this infrastructure for intelligence, but it also, what they were really revealing. And I think if I'm going to give them some credit. Why there's no dates about when any of this is going to be available. And it seems like maybe they just said, oh, let's catch up and give an announcement, but we're not going to give any dates because this could take a year.
This was a developer conference. So what they were saying was, look, here's a bunch of tools for all the developers who have apps to go out and build Apple intelligence into your app. So what does that mean for us? There is this concept earlier this year that was referred to as large action models. Um, remember, I think it was like, um, uh, the. And right after new year's what's the conference in Vegas. I'm sorry. I'm such an idiot. Um, well, there's the name show for music consumer electronic show. Yeah. Consumer electronics. So the, the big runaway hit of consumer electronics was the rabbit device. And on, and Matt, like I just, I, the second I saw it, I was just like, everybody just stop it.
And, and, you know, I'm a bit of a cynic, but what it was with this little brick that was basically opening, uh, open AI, but with the ability to like run actions, uh, there were two reasons why I was so dismissive about that. Well, a, you can have your own, we can do act. We can have open AI run at API end points already. Yeah. But the real reason why is because it was this brick. Well, first of all, it's sold out. It was the most popular thing. People were freaking out. Okay. Well don't, because it was a complete waste of time. And the reason why it was a waste of time is because you would have had to carry two things in your pocket. Now this dumb little rabbit brick, and then your phone.
What Apple just did. Well, first of all, literally anyone who has one of those rabbits, go throw it in the garbage. I'll wait. Okay. And here's why, because now you gotta wait until fall though. Well, but the point is it was always coming. It was always going to be that you could do the same thing. And what is this that we're talking about? What we're talking about is, um, not only be able to ha it's going to have access to all these apps and it can orchestrate these things that we're talking about. So when we're talking about like looking into my emails, my text messages, see. When our friend is getting back from her vacation, those are all Apple apps.
The big thing is going to be when you can do things like saying, um, Hey, Matt's coming over for dinner on Saturday, go check, um, open table to see if our favorite restaurant has, uh, availability at 10:00 PM. If not, then go find off of their menu, our favorite appetizer, and then go into Instacart, another third party app and put in all the ingredients to make this at home, fill up a cart and have it scheduled to deliver to my house. That. Apple can't couldn't yet reach into those applications. Now it can, that's what, um, loosely is what app intense is. They're essentially, um, APIs that, uh, API wires that are hanging out of all our apps that we can build.
So there's a bunch of them. There's a natural language API, a vision API in a speech API that they announced it, uh, that we have availability to not we, but like app developers do. And I want to be clear. We're not actually app developers. I know we like. You know, yeah, like hijack the term app relational database developers, but anyway, probably should have never done. Yeah. So app developers using Swift can do this. So what can Claris do well, aside from the natural language endpoints, the vision endpoints and the speech API endpoints, which are literally API endpoints currently, my understanding is, and I'm, I'm getting into the area where I could be wrong in a bunch of stuff.
And I just haven't had time to, um, like rabbit hole. Sorry about that. About the rabbit device, but rabbit hole into these things, if they're just wide open endpoints and we can just connect to them from FileMaker. Cool, done. But I don't have any evidence of that right now. Yeah. I believe these are through Swift, but here's something amazing. Oh my God. Okay. And you, you can laugh at me when I'm totally wrong about this, but let's just have some fun and pretend that I actually discovered something. So, so look at these models on screen. I'm showing a list of models, whisper, uh, which is, uh, opening eyes, open source. I'm showing a model that allows that translates speech into text inputs and, and text into speech.
That's how we actually talk with the app. Right. And that is totally available for us as FileMaker users to it's a open source model. You can just go on GitHub and use stable diffusion, of course, is the diffusion, uh, model creator one, the rest of them, Mistral, uh, Lama Falcon. Those are all open source models. We saw a couple of those on those benchmark lists. Those are ones that you can go download and run locally on your device. And I, I would say. Okay. Okay. Like open source is all like, you don't have to pay for those token wise, but you definitely have to pay for compute. And then there's Clip, Quinn and OpenELM. What are those? Well, go read the article about things that Apple dropped before this announcement.
Those are models that they dropped and that are also available. So what all these things have in common? Matt, we can use, or we'll be able to use, CoreML has a tool called CreateML. Okay. We have talked, we've danced around this topic in the past because in FileMaker, you can go train your own model and then at the end, compress it into this thing that's called a .ml model file. Does that sound familiar? That's come from the early days stuff, like before the recent crop of AI stuff. FileMaker 19. Yeah, exactly. CoreML stuff. So the .ml model files are meaningful to Clare's folks because you can take those models after you create them and drop them into. FileMaker container field and with some supported native script steps with FileMaker, like run them, make them available in that user's session.
Huge deal, right? Chris, is this the first podcast you and I have done since FileMaker 2024 shipped? I think it actually is. No, we did one with Ronnie. Oh, yeah, yeah, yeah. We did a huge. That's right. That's right. That's right. I did it at the conference. Because anyway, my brain was connecting things weirdly because of. Our next topic when I'm talking. Yeah, we did a 90 minute one that everyone should check out because Ronnie Rios, the product manager of FileMaker 2024, came and went through line by line of all the different features. So, yes, I know you just teased me. So what does this mean? Because this is a big deal. OK. They said that now you can take any one of those models that we just listed off, run them through the CoreML tools, which is which is both the the create ML tool that is the part of Xcode and also there's a command line interface that you can run from within a FileMaker script script to run CoreML tools that will take those models and make them into a dot ML model file, Matt.
So imagine if this means I can actually take Mistral 7B, turn it into an ML model file and drop it into a container field in FileMaker and run that. Send data to it and get data back. If this discovery is what I think it is. Oh, my God. And that's going to be Apple only because some of the CoreML stuff was Apple only. It all is. So, yeah. So, so I guess I was thinking of a Mac versus iOS, but yeah, Apple. Yeah, yeah. Well, a little bucket of water on that. But yes, I think people need to know that that the reason that we don't hear a lot about CoreML probably is because it's Apple. Apple only. But get but but think about this instead. On device, you've got a FileMaker app that you want to run through the SDK and manage in an MDM for business use cases on device that currently has access to these CreateML models.
So you've seen demonstrations. I got a bunch of stuff on the Internet of like creating image or vision models that recognize things and you point your phone at it in your FileMaker app. And you go, what is this thing? And it goes, oh, it's a spark plug. And here's the record in your FileMaker database for that. We've been able to do that for five or six years. Yeah, I remember that demo. So imagine that actual thing. But using Mistral or Falcon or Whisper or, you know, some of the other models that we just looked at, that's how I think people should be looking at this. And I think that is potentially very profound, whether it's just for Apple or not. But, you know, it could also just be for devices.
Right. That also communicate with FileMaker server through non-Apple environments. So TBD on what that means. And by the way, just, you know, and then we'll get to your announcement. So to put a bow on this, the fact that CoreML is Apple only is what got me to go look for cloud-based API models in the first place, because I was doing things like summarization and sentiment analysis and keyword extraction. And I thought, well, I can only do this on Mac. So I bet you there's a cloud service out there that does it. And oh, great. They have API access. Cool. So sorry, CoreML. I don't need you anymore. I can do cross-platform model integration this way. And that's ultimately what led me to GPT-3 as well, too.
So thank you, CoreML, for not being cross-platform. Okay. So lots to check out. Check out the link in the show notes of some of the stuff we've been talking about here. Okay. So, Matt. Next topics. So I was at the .FMP conference, which happened, I think, in Berlin, which happened two days after 2024 shipped. So all these people had sessions, and they couldn't list what it was for. But a lot of them knew what was going to happen. They didn't know when it was going to happen, but they were able to actually change their session description and then show them new stuff. It was awesome. It was like such perfect timing. Yeah. I think, actually, so they updated their, this, per what you're talking, the timeline you were talking about before, they were able to update all their stuff when the announcement came out.
But it was still a week later that you went to the conference, which is still like a week. Are you kidding me? Everybody had all this stuff already. No, it was really only a couple days. The first session was like, I think, literally two or three days after. It wasn't like a whole week or something. Yeah. Three days after the launch, you and I did our interview with Ronnie Rios. Right. And you had not given up. And I was at the conference. I was at the conference. Yeah. Oh, yeah. You were at the conference. Perfect. Now we're putting it all together. And this is kind of why I forgot it, because I was in the middle of, like, we did that one, and that was like, you know, 3% of the stuff coming at me that week.
Of course. Because I was in all these other sessions, right? So, and by the way, you did some great interviews. If people want to look at our bonus episode from last week, you can talk to some folks about that. But is, by the way, was that the last Berlin conference? Why? What's, I don't understand. So, the guy who runs it, actually, the very last, the guy who runs it, the guy who runs the last recording of the last episode, he talks about it, exactly why. But he had been running it for, I think, 12 years, and all alone. Like, it's just his deal, his baby. And that's a lot, right? Agreed. So, the other conferences that happen in Europe have teams behind them, or even, like, multiple companies for the EngageU conference.
And the EngageU conference last year had 200-plus people, and they're aiming for 300 people this year. Wow. And so, that's by far the largest English-speaking conference in Europe. Anyway, so, at this conference, there was so much awesome AI stuff. It was great. And there was a lot of other really good things as well. Kevin Frank was there showing some amazing stuff. That guy's kind of smart. What was your favorite, like, takeaway? Oh, man. Just, like... Too many things are coming. I'm kind of jumping in my head right now. I think it was actually some of the simplicity, some of the stuff that looked simpler that maybe isn't quite so simple for the new script steps that we have in 2024.
Right? The embed and the perform semantic find. Yes. Right? Those two are the key. And there's one other one, which is, like, basically set it up in figure. They're essentially companions with each other. Like, you need one performance search. Yeah, you configure it, and that's the one you set up. You send the username and password. And that's the only time you send the password. And then the subsequent calls either embed or do a find. The embedding is an API call, and all the perform finds happen locally. So, basically, seeing all those demos and used in different ways and how short of a script you can make was kind of the big thing that I saw. And I saw many different uses of it.
There were some great discussions. But then I've been, I guess, I'll just tie this into the next thing, right? Please. So. And for some context, we've been talking about the idea of semantic search in a lot of the episodes. Yeah. And one of the personal challenges is, well, you're very well known for having a search. I'm not going to call it a product, but it's a, you know. Well, it was. At one point, it was for sale. Right. Okay. Yeah. It's a completely. Well, it's called FM Search Results. I just call it Search Results now. It's version seven is the most recent release, which is available for free on my website and a bar that training. I still have yet to see a database that I think would not benefit from having it in there because it can search across all the tables and show you results on one page.
And now, were you originally doing this in the virtual list era originally? Or what was the evolution of your technique? The first version of it actually was a virtual list. Okay. Actually, I don't know if it was that. No, it was even before that, actually. I think it was actually repeating global field with the display. So the search stuff happens at the back end, but then there's different widgets to display the data. I feel like I saw you. Did you present at the DevCon in San Francisco? Yeah. On this? I did. I think I saw your session and it was maybe. I thought then you were at the virtual list level. It was like the first time I saw somebody talk about virtual list.
It still is for the current version. Okay. Because it's a. But it's. It's. It's. It's showing JSON data. So you can basically take the JSON object and show it with whatever way you want. But the virtual list works really well. You know, a few hundred records into a table that have no data in it. And it just basically grabs a line number and says, show this information. And when you click on the line, it knows that it knows the data that you're viewing and the metadata of like what table of that, what's the record number and then navigates to that particular thing. So that's what's that's what's really cool about it. You can search across everything and it's wicked fast.
Okay. Amazing. That's right. That's right. I just dated myself by saying wicked fast. Well, you also dated search because you were talking about virtual list. So, so what would be the modern and we've talked about this in our conversations, the modern adaptation of that would be to integrate semantic search somehow. Yes. And you took the personal challenge for that. That's what you, the gauntlet you threw down and I'm, I picked it up, um, is to, uh, to basically begin rewriting it to, in addition to the other one, to use semantic search. Awesome. Or semantic find. Yeah. Um, so, uh, yeah, I guess technically it would be semantic fine because we call searches, finds and file maker.
So I have a lot of questions. I would actually like to have some conversations with people about the best way to do this, but kind of what I'm thinking is this in order for semantic fine to work, you have to have an embedding, which is a, you have to stick a blob of text into a container field. You need to first convert a blob of text into actual embeddings, which are mathematical vectors. That represent that in a multidimensional space. So imagine if you will, a, uh, a typical CRM type system, you have a company record, which has four employees, there's 20 invoices, there's a bunch of notes and there's, you know, maybe just that, right. Okay. Um, so you, you take that record and basically create a Jason object of everything that has meaning embedded in it, right?
So you can use types of Jason that actually say, these are invoices. Um, and you can, and maybe even have some other stuff in there too. And the, the meaning you're talking about is you're providing context in the structure of the Jason object. Yes, exactly. By the naming convention, which is, I didn't think about, but that's kind of cool. So, uh, it's actually a little bit of a pain to do that for a whole entire set of records. You have to kind of loop and there's some wild stuff. So, oh, because you're constructing an object. They're not just like, go search for these results. You're actually taking fields from very potentially. Different contexts actually, but you're assembling them together as one Jason object.
So the way that I'm, the way that I'm thinking that this was going to work is this, um, in each table, when you edit a record or create a record, I think really edit, because when you create, there's nothing in it, it would, um, it would, uh, create a Jason object for that information with all the nice stuff and then go to another table, a table dedicated for, for, um, search the, that's the only place where your container field lives. Just to be clear, the container field, which is the, the file maker 2024 binary that allows you to store the vectors, which is very meaningful because if it was text that you were doing, your indexes would be so insane and your performance would be ridiculous.
So they have to be these new types of containers, which is not really big, that much bad. So it's, it's, if you store it as text, it's worse, the files bigger, but it's not so bad. It's just definitely the best practice is for sure to do it as, as a container. If you had like, um, a database with like a thousand meeting transcripts in it and you wanted to search within them and you turned on text, like it would be. Well, so the other, so the other idea for my approach is, uh, so what we're going to do is containers, not, not throwing bucket water and saying these containers are a big deal. That's awesome. Yeah, definitely. So, um, so it goes into this dedicated table that's flat.
It's going to have one record for every record and every table in your database. That you want to search against. Awesome. Okay. So in my example, you'd have one, it would have a record for the company. It'd have five records for the contacts, 50 records for the invoices and 30 records for the, for the followup contact notes or whatever. Okay. Sure. And then also, you know, there may be an address table and a phone number table and some other little small ones. Yeah. Like basically child table. Yeah, exactly. And, um, and then maybe the little tiny ones, it doesn't necessarily have to have a record for each thing, but for, for like a, an invoice. And all of its line items that would be like a record, you'd have a Jason object that would have the information.
And then interestingly, you could create an array for the small ones, as you're saying, like the addresses and phone numbers, you could just do an array or something. Yeah. So basically you make a one Jason object and interesting in the way that it works in 2024 is you have to have a field in the table that has the Jason thing, or you should do it in Jason, not text format, um, that you have to then embed. So the field and the embedding container have to both be a field. You cannot. You cannot, unless I'm mistaken, embed from a, from a variable. Oh, yeah. I hadn't thought about that. I don't. Yeah. You know, maybe you can, I did it with a set of records.
There's two different ways to do it. You can embed a single record or you can do about embed a set of records in a loop through and do them in chunks. So you were, you're hoping that you had variables so that you could loop through and just create one variable to embed rather than have to do individual records. Yeah. So, so my thought is, so then when you want to do it. Uh, a search, it would navigate to this table because another key thing about, about, uh, semantic searches, you must do the search from the table that you're in the same context. If you do a related search, it's kills the pro the performance because the it'll automatically do it server side and it'll be, you know, hundreds of times or whatever faster because the server has insanely fast access to all of the data in those container fields.
I was thinking that for it to read the data in the container fields, it would be slow. No, it's surprisingly fast, except if you have like 5 million records in your, in your set that you're searching against, that's going to be slow. So you're saying the preference is local versus server, as opposed to our normal thinking where we do P sauce and we push it off to the server because it has more processing power. Yeah, yeah, yeah. Something like that. So, so one of the features that I had in search results is before you do a search, it actually determines what it is that you typed in. So if you type in one, two, three, four, five, it goes, oh, that's an integer that could either.
Be an invoice number or an account number or a dollar amount. So I'm only going to go look in those places. I'm not going to go look for, I'm not going to look for one, two, three, four, five in a text field that contains notes. If you type in something that's obviously an email address, it's going to look only in places where an email address can live. If you type in something that it, that it, that looks like a name because there's a name lookup table, it looks only in the place where names exist. And then also does name substitution type stuff. If you type in something that looks like a name because there's a name lookup table, it looks only in the place where names exist.
If you type something that doesn't, doesn't match any of those things. And there's several, there's date, date, range, integer, decimal, blah, blah, blah. Then it goes, okay, well now I have to up the game and I'll just assume it's text, but we could even, so, and then I could do a search for text. And which says, which means I can either do a lexical search or a semantic search. Right. Yep. And then I can also look at only a small set of records. So for example, if I, you know, the. The, the, the code that runs it doesn't necessarily have to look at all million records in that table. It can say, oh, I know that what you're looking for, you started your search from an invoice layout.
So I'm not going to look at the entire set. Just your related invoices. Yeah. So that one of the, one of the other records in that, one of the fields in that table, that's a dedicated table would maybe be like table number or name or something. So I would say, okay, find all the ones that are indexed there. And then if you're looking for all the ones that are indexed there, you're going to have to look at all the ones that are indexed there, which would be invoices, which would be fast and then search only against those or all the ones that are invoices or line items or whatever. Well, we were kidding around about our, in our context episode earlier this year, where we said, it's not the, it's not the context that you think, but you're actually doing a very clever use of the context that you do think it is like us file maker people context to establish what, where you're then doing your semantic searches.
And I think that's actually sneaky brilliant. We'll see. I've had, I have. I haven't done that much testing on it yet, but then the other thing I'm thinking is what if, what if the user types something in that they're now going to expect, which is who are my best three sales reps in quarter four last year? You know, that's not something, you know, the data's there. It has the invoices, but I don't know. I don't know how that's going to work because that's not a search. That's like a report. That's a, that's a reasoning. Well, here, I have a thought for you. Here's my initial instinct. We talked earlier about. Personal context and like when you're writing your prompts to speak to language models, you got to tell it who you are.
Another thing you have to do is tell it who's going to consume your information. We often refer to those as audience personas. Right. And, and also then the rest of that is where you're providing instruction and, and giving it information about what is a meaningful result. One of the things. So one of the things that I'm zeroing in on. You said best. So, um, we literally have customers who've come back to us with this and, uh, I've seen people out in the wild talking about this and they go, I just asked, I don't know, chat GBT, what's the best, you know, Mexican restaurant near me or something like that. The problem is. The first thing these models are going to think is, well, what do you think the best is?
Right. And so that, so I don't think that's a miss. I think that's an opportunity to define within your, either your user persona and, or your audience persona. What. What best means to them. Right. And when we're writing user personas, we will literally say the way they utilize their data is they think that this is the biggest, like we did, uh, um, uh, we did a, uh, uh, uh, GPT for fantasy sports where I connected to this, you know, this fantasy football API that I have. And instead of saying, who's the best player I defined on this like page of instruction, what best means this is better in this case, this column, when it, the higher is the better in this column, it's this.
And, and, and whenever you're deciding. The better, sure. Exactly. So in your case, you could have profound, amazing things happen if you are able to define what best means, for example. Yeah, exactly. And we do, we all know that because we know what our users workflows are and what they care about and all that kind of stuff. So, um, yeah, I think that's an opportunity. Okay. So the embedding that does the Jason, this is the other one of the other key parts where you have to do one AI call that actually, well, that's one API call that you have to do that sends that information. Out and creates the embedding and then puts it back in. You can actually have that done on the server, on your loan.
It could be in house. Um, and you could run that on, yeah, your FileMaker server or some other server in your building, but, but it's also super easy to do with the, with the, uh, uh, open AI API. Yes. Well, I will say just for some context from out in the real world on that, um, like we just did a, uh, for one of our FileMaker customers, massive embeddings had nothing to do with. It had to do with pairing and matching. Um, I won't reveal what they were doing, but it involved like, you know, 2000 records in a FileMaker database that had huge, a big document that we extracted text from with the language model. And then we embedded that big block of text, kind of like what we were just talking about before.
And we used open AIs, uh, large, which is, um, the reason we did that is because it was so much text. We needed the dimensions. Yeah. Like you have embedding models that have like 300 features or dimensions. Um, which means. Uh, we did a thought experiment where I said, Hey, define a bunch of features of the word soccer. And you gave me like five. And so then we said, well, a language model can do 300 or up to 4,000. And that, that's what, what, why we use embedding models instead of humans to do this. Right. But on the 8,000 dimension embeddings model against like 2,600 records, it took like two hours to run that on a server. So like, like these are no joke.
And, but it didn't matter. Because we were doing a set it and forget it. And then we can embed one record at a time in the future. And it only takes seconds, not hours. Right. Yeah. This, I think I, I don't know how long I ran it for, but it was much, much faster than that. But the point is if you were to use like a, um, a haiku, which is the three parameter billion parameter version of the Claude Opus, for example, which is what we did, it takes minutes then. So the point is the model, the embedding model selection you're making not only has to do with speed, but the number of dimensions. And I think that's really important. I think that's a really important thing that people need to take very seriously when they're making those decisions.
Absolutely. So another concept that I have is since there has to be this one API call, well, why don't I just do an, um, why don't I take that blob of Jason and do a API call that says, take this Jason and using what you know about me, create a bigger Jason blob that describes exactly what this is optimized for semantic search. Great. Then you take that and you embed it. And now you have, now you have. Uh, a much, much better thing, uh, to search against, I think. So this is, and you're talking about using what you know about me is your user persona that you crafted for that individual user. Exactly. Exactly. Or the system. Yeah. Or the system. So this is work, right?
This is, uh, so by the way, user personas are the system for that one user of your system, of your applications, API call. Um, you know, a lot of people just like put in, that's where they put you are a helpful assistant, but you can put, I mean, we have so many tokens available. You can put tons of information about this person, by the way, you could, to your point, imagine if you said you're searching against a customer table for invoice information and, and you know who the user is. It's Chris is in your system and he's about to execute this search. Guess what? You also have a log of the last 1500 searches that Chris did. That's true. You could, you could actually incorporate those as context to determine, to make some of the decisions that you're talking about it.
Like he's. Often found that these are the type of, he only cares about California customers or whatever. Like you can model. And provide that information about me into that search in the manner that you're talking about. And that's a big deal. Uh, not with the perform semantic bind feature. So there's maybe other ways you could do that, but I don't think you can send all that information. At least I just, I haven't played with it that, that far yet. You mean to embed or to, well, the embedding can know that, but the embedding is there one time, right? So you're embedding the information for a company unless. Unless you're embedding, unless you're embedding that one array of a log on domain on the fly and then coming back and doing it.
And by the way, that's how chat bots work. So, so think about this for a second. So a chat bot with retrieval, what you've already embedded a whole bunch of documents, like a bunch of HR documents. You did that weeks ago and maybe that's a set it and forget it. But then when the user is asking a question and they type the question into the chat bot, you're actually. You're actually embedding that question real quick, boom, shooting off and embedding it and then doing a semantic search. So it's the same thing you're talking about. You can say, yes, they're asking a question. You just go, boom, go, go embed that. Then you could do a semantic search based on that.
You're absolutely right. I forgot. I, I, I didn't connect that, but you're right. So before you do the search, the, the, the string that you're searching against. Um, because there's two embeddings, there's the input or command that's being embedded. And then there's the reservoir of stuff you're, you're querying. Against in your vectors. So this is going to be some really fun experimentation with search results. Like the way that I, what really, really makes me happy when I do stuff for the community is not just to do something that's technically cool, but to do it in a way that's really light and fast and easy to integrate. That's where most of the work for like FM log and FM search results went, um, so that you can integrate it in an hour and then it just works, you know, anyway.
So that's my goal. Amazing. Well, so you're speaking at a. Conference this fall in Europe somewhere. Yeah. Two of them. So October in Rome, November in, uh, Sweden, I think. And these are the ones that you're going to, uh, show in your presentation. Yeah. So I'll be talking about this. Um, and I'll also, I'm also doing classes, uh, like a day, the day before the conference, I'm going to be doing a paid class. So you can, oh yeah, that's right. That's right. You should put that in the show notes, uh, by the way. And, um, and a quick couple of plugs and then I'll just end with like a. A little two minute presentation here. Yeah. We have two updates from you. Really?
Yeah. So, um, learning initiative and also your intuition B3, which I guess, so, um, Claire Claire Stottle, which is what the Claire's AI learning initiative. We, we somewhat tongue in cheek, but I think in a fun way named it Claire Stottle. Okay. Whatever. Um, so, uh, that was a shower thought. I was just like, what should I, oh my God, Claire Stottle. So, um, on July, yeah. Uh, on July 12th, we're having a follow-up, uh, if you look in the show notes, you can join the discord channel where we have active conversations about it. You can see our initial kickoff that we did last month and see a lot of the conversation that everybody's having. People are already registering for the July 12th, 8:00 AM Pacific standard time, um, uh, meeting it's open to anybody that wants to come and talk about it.
Mostly what we're going to talk about is did anything that apple announced and, you know, our Claire's friends will be there too. They've been very supportive about this. Is there. Anything that we could use in this initiative there and, and there'll be some Claire's folks there to, to provide their thoughts on how this might work. Um, and it really, we just want to keep the conversation going. So I strongly encourage people to check it out, go to the show notes, go at the community. We've been talking about just Claire Stottle. There's no other term anywhere you look for it. You'll find the discord information that that's part one, part two is before you go to the part two, when are we going to see a proof of concept of that?
We can start playing with. Well, so I thank you for that. That's an important question because what we're actually trying to create with this is first step is to create a, a, a community driven knowledge base of, uh, a higher level of Claire's related file maker related knowledge than the latent intelligence that's in the model. We have to do that first. And so we're talking about doing that as a knowledge base that we can then go vectorize and make it available to different models. So once that knowledge base is there. And once we have the ability for people to take their own models or their own applications or whatever, to go have access to that knowledge and then to embellish the latent knowledge within the models, there'll be all sorts of products that are coming out of that, that are going to be really amazing.
So, um, and that's part of what we need to talk about too, is like, cause we're not really talking about building a model or training model. We're talking about building a knowledge base. And then what happens after that will be very interesting to see what we do with that. Maybe someday we fine tune with it, but right now it's a knowledge base. We're vector retrieval, um, that's step one. Um, for me, in addition to the Claire Stottle on July 12th, I am really excited because I'm speaking at a conference in my hometown of Milwaukee next week, as we record this, uh, I'm from Milwaukee, Wisconsin. And one of the things that, uh, was there my whole life, this thing called summer fest, which is like a music festival.
And it is a big deal in my little town. And a few years ago, they started a tech version of it. They're, they're in partnership with South by Southwest. If you remember South by Southwest was a music festival. Then it became a movie and tech one, they're trying to follow in the same footsteps at the scale of Milwaukee. And I'm so proud to be able to go to my hometown and talk about AI. I'm doing a refresh, a very significant refresh of the session that I did at engage, where I was just profiling a bunch of real world deployed AI things. Only a few of those I'm showing again. And I have a whole new set. I have a whole new hour, you know, kind of as like, like comedians talk about, I have tons of content and a bunch of stuff that we've been doing since then.
And. And one of those, um, things that I'm going to profile, not all of them, but one of them is, um, this, um, intuition platform where we just released the version three, what the users on YouTube are looking at are, uh, custom chat bots. So we have an entire execution environment. This is like code interpreter for, um, you know, the code, uh, uh, chat GPT has code interpreter, which is where you're not just chatting, but you're doing visualizations and Python. So we have. Over 480 Python libraries available, built into this application. You can obviously do chats. You can do visualizations. You in, in the biggest deal is, is it's multi-user totally secure, meaning every user gets their own user persona.
All that modeling that I talked about from a user standpoint is happening full customization. These are just the prompt template builder that you just saw fly by very quickly there. But the, the point is it's all connected to your live data sources. So unlike with code interpreter, where you have to upload a spreadsheet. To be able to, um, to be able to interact with it, we're connected to like right now, what you're looking at is, uh, somebody connecting to their entire file maker suite of applications through DAPI in this case. Right. So what we're able to do is, is, uh, containerize this and drop this into people's, uh, work environments and allow them to enter, keep all their workflows the same.
We're not introducing new stuff that you have a new app that you have to work with or anything. You can actually build this functionality right into your existing file maker systems, your SQL systems. You have access to API data, all multi-user, all totally secure SOC 2 compliant, um, ISO, uh, compliant. All the security check boxes are checked and model hot swapping. We, uh, we're able to add the GPT-4 Omni within 45 minutes of its release with this. And so that meant, and then we push that out to all of our, uh, all of our customers that are using this as an environment. Um, and the way to think of it, when I say environment and execution environment, think of it as it's a operating system that you can run apps on, right?
So we all know operating systems. We all know apps. Ours are, um, an environment that you run assistance on. So we're looking at a training assistant that, that a user is connected to an actual file maker database with a bunch of file maker data. And they were able to ask, give me a chart that shows the progression of training and give me, um, all the sessions completed by certain individuals. Like this was all, and then you don't have to say you want a chart. Even you can just say, go find this data and present it to me in the most compelling manner possible. This is our version. It's free, super excited about it. Um, this can be deployed in the browser since that we're showing on YouTube, or like we're doing a deployment for inside zoom chat.
You can, we've done one in teams. You can do one inside Slack. You can do it inside your file maker application. You don't even have to touch your workflows to be able to do this. So I'm super excited about it. Um, and I'm also excited to be talking at that conference, uh, next week too, just because I'll be, you know, staying at my folks house and, um, you know, being able to speak at a conference at the same time, which is just crazy. So. Um, just from a, you know, a personal worlds collide thing. So more to come on that in the future, uh, I solutions, ai.com/intuition also in the show notes. I'm happy to chat with people about it if, if, uh, they're, they're curious, but a lot of it on LinkedIn as well.
Sweet. We did it. So we'll, we'll be back. Um, maybe not in a couple of weeks, just because the 4th of July, we don't know people put us in years, but right after that. Um, we'll, we'll be back with another fun episode of Claire's talk and AI information overload.