navarre.aiClarisTalk AI · the FileMaker AI podcast
Are LLMs Just Databases? The Real Story. Plus, Apple AI Predictions for WWDC
Predicting Apple’s AI Play: https://medium.com/@isolutions/predicting-apples-a-i-play-719b1c2f91a2 LLMs are not Databases: https://www.linkedin.com/posts/crisippolite_ai-activity-7185307614539329536-qVcm Links: Claris Community Live: Learn about AI prompting basics Thursday, May 30, 2024 8:00 a.m. – 9:00 a.m. PT https://content.claris.com/e/526331/filemaker-ai-webinar-may-30-em/9tt3jn/2284491076/h
Apple PodcastsSpotifyYouTubeAll episodes
Transcript
Transcribed by machine (whisper, large model), 11,276 words. Names and product terms are sometimes misheard; the audio is the record.
Claris Talk AI. Podcast time, Chris. Hey! We've got a few things on the agenda today. One, we're going to talk about our main subject. Then we're going to talk about some of the OpenAI announcements for Omni. Actually, there's a ton of announcements in AI. There's a ridiculous amount of announcements in AI. Then the other big thing is we're going to be talking about Apple and the upcoming WWDC, WWW, and what we expect to see, what we want to see, and some really good evidence for what we think we're going to see. First off, as a database developer and as not an expert in all this AI stuff that you are edumacating me on, in what way is a large language model a database?
Is it flat? Is it relational? Because obviously it's a database, right? Right. So, excellent question. Excellent question. Nice baiting. No, this is a super important topic and concept. It's because, well, let's frame this up a little bit. First of all, anybody who's listening to this or watching this is a Claris person interested in AI, right? What do all of us Claris people have in common? Well, we're all FileMaker people. We've been building databases. In a lot of cases, decades. I think you're going on 70 years or something like that, right? Dog years, of course, naturally. And then the other part is, and this is what I really want to zoom into here, is the overall perception.
Not the people that are calling us up and asking to build AI solutions, but the people who aren't quite there yet, they think AI is a question-answer machine. Right? So the concept is almost like, yeah, it's like this source of knowledge, and I'm asking questions, and I'm retrieving knowledge from it, which is exactly what a database does, right? Kind of what Google does, too, right? It's literally what Google does. And also, that might add to the blurry lines, too, honestly, Matt, is that all this data is the same data that Google has indexed for searching purposes, right? It's all the web corpus of data. So, can you ask questions of it? Can you retrieve information of it? Sure. Yes, of course.
You can. Is it a database? No. And like a hard no. It is absolutely not a database. And here's some really important characteristics that make it not a database. First of all, if you said, hey, here's all the passwords to my FileMaker database, and you type that into ChatGPT, and then someone else came back later on and said, hey, what are Matt's passwords that he put in Chat? It's that there's absolutely zero chance they can retrieve that information. And the reason is because those models that are actually, quote-unquote, too late, right, that are answering the question are they stopped learning. They stopped training when they were published, so to speak, when they were released into the wild.
So, when they talk about don't put your sensitive data in ChatGPT because you don't want it to be part of the model, they're absolutely not saying that that information is going to exist in the exact state you put it in, in the model, ever in the future. What they're actually talking about is training data. And the thing that's really hard, I think, for people to understand is that these aren't databases. In fact, what they are, are these amazing learning, like, machines. They're not question-answer machines. It's not their intent at all. Right, right, right. They don't retrieve information even though they sometimes can. Early on, I actually did a test. I said, who was on the cover of People Magazine in, like, July 1960?
It doesn't know. Well, I mean, I would almost bet, yeah, I would bet that it could know something like that, right? Because that information, is that it's training data. And if you're just asking, so when you ask a question of a transformer, yeah, well, actually, it's even, I think if we even zero in even further, it's kind of interesting what's actually happening. When you ask a question, so what a transformer model is, that's what the language models are, they are probabilistic completion algorithms. Like, super incredible probabilistic completion mechanisms. And it's very much the same thing as when you're typing into, like, Gmail or something and it knows the end of your sentence. It's, like, the most probable thing.
And I think, actually, in the Gmail case, or at least for sure now, in, like, the Google, you know, AI space and the Microsoft AI space, they're looking at your emails and saying, this is most likely what you would say, right? Even in the search bar, this is most likely what you're asking based on your search history or something like that. That is a probabilistic completion algorithm. So, when you're asking a question, who was on the cover of People Magazine in 1960, you're not actually asking a question. You're typing a sentence that happens to be interrogative, and then the answer is really not an answer. It's just the most probabilistic completion to that question, right? So, sometimes you're going to actually get the right answer.
Sometimes you're going to get the wrong answer. The whole point is, and this is where I want to be aggressive about this, is that is not a source of truth that you should care about. Let me propose a thought experiment to you as a 70-year-plus in DogEars FileMaker developer. 41, actually. Actually, in real years as a database developer. Very impressive. That's very impressive. Yeah, that's hats off, my friend. So, let me ask you this. In your 41 years, how many times has one of your customers asked you to integrate Google searching into one of your FileMaker layouts? How many times? Let me think now. Zero. Exactly. Or even question answering. Like, ask a question of a repository of knowledge. Zero.
Because the database, this whole point, find mode is literally how you query your own data, right? So, we already know. You're looking for what you have. What you have is a source of truth, not what. Bingo. So, we should not care. Like, we not should not. If you're doing it right, if you're thinking this through correctly, how can I leverage AI for my customers or in my own databases? You should not care about the information that is stored inside ChatGPT. Or any of these models. I'm sorry. I want to try to not make AI seem like it's ChatGPT, but like any of these models, we don't care. So, instead, what we're going to do, and we've talked about this, you know, extensively in previous episodes, like the one on context, where we talked about retrieval and inserting your own truth.
We want to be responsible for the truth as developers. So, let's just say, as FileMaker developers who want to integrate AI, one of the main things you're taking responsibility of is you are the one who's in control of truth. That way, when people ask a question, you're going to go ask that question of your FileMaker database, or maybe documents that are stored in your FileMaker database, get the answer. Then, you're going to go to the language model. And you're not asking the language model a question anymore. All you're asking the language model to do is potentially analyze this. Summarize, talk like a human. And honestly, if we looked at language models as...
Like, imagine if I told you a new version of FileMaker was coming out, and it had this add-on, or this function that could talk like a human. Like, five years ago, if I told you that... Yeah, or not even, like, out loud, but, like, you could take non-sequiturs of information, like a found set, and have it come up with, like, and make a paragraph on demand. If we told people that that was a new feature coming out in, like, FileMaker 17, people's heads would have exploded. That's what this is. That's how we should be leveraging what this is. So, and the other... And the reason why that's so critically important is because I don't think people understand... And as developers, we need to take ownership of this information.
People don't, like, understand the concept of how this works. How they're trained. Where the truth comes from. And that is why there's so much focus on hallucinations, as well, too. Right? Quick anecdote. Remember one of the first videos you ever posted on YouTube? Of course. Yeah, you did a really, really good ChatGPT one. I was so proud to see that up there. More coming. Yeah. Awesome. And one of the things you did in the video, and maybe if we think of it, you can find the link and put it in the show notes if people want to check it out. But you weren't, and we sort of disparaged this idea of zero-shotting, which means you just ask it and then think you're getting the final response back. Right.
That's not how you should be using these language models, either. But you said, hey, I don't know if it was a script or a calculation. Yeah, I think it was a calculation. I hadn't really done much with scripting at that time. And so you said, what's this calculation? And then, it wasn't, like, perfect. So you said, hey, no, actually, it should be blah, blah, blah. And then you got, like, a pristine, good calculation that you could do something with. Yeah, I did one on SQL codes, too. It, like, writes ridiculously good execute SQL. Really, really good at SQL. And, like, one of the comments, and I mean this with love in my heart, I don't even remember who the commenter was, but it was just notable that one of the comments was way to go, Matt. Now you just taught ChatGPT not to make that mistake again. Right.
So that is where I think people think it's a database, where people think you I thought at the time, by the way, I thought at the time I was actually updating it with my, yeah. Right. And now, just to be clear, you were updating it in that one conversation. Right. You were updating it in the token window of that one conversation. Yep. But the second you start another conversation, or another user starts another conversation, or yeah, even you in your own account, and you start another conversation, it has no knowledge of what you said. You can store that memory somewhere, and then retrieve that in other conversations if you want. We can do that as developers, but the point is, the model doesn't change. Following up on that anecdote, the error that it made in the calculation, like when I do calcs, I only ever use case, I never use if.
And so I said, don't use if ever, just use case. And it did it for the next bunch of them, and then later on in the same conversation, it started using if again. I'm like, what did I just tell you? And so we exceeded that window, right? The token window. You exceeded the window, and at the time, it was probably only like eight thousand tokens, or something like that. And now we have like hundreds of thousands of tokens, so it's less of a concern. Or later in the episode here today, we'll even talk about million tokens, or two million tokens, and what that means. But the idea is, we can affect the knowledge in a model, and there's two ways to do that. One of them is through something that's called fine-tuning, and I just, I'm here to report, I've been a rabbit hole on fine-tuning. Everything there is about fine-tuning. We've tried it. My team has done this. Luckily, we had very patient customers in the early, early years of this, like four years ago. And we tried to change the knowledge in the base models.
The amount of data that you need to actually, and it depends on the use case, but just to give people an idea, the amount of data that you would need to actually affect the knowledge inside the model, a copy of the model, it's not even the real GPT-4, it's your own copy of GPT-4, is more data than I think almost all of the FileMaker databases we're talking about. We're talking about you would need to have pristine, formatted prompts and completions, and hundreds of thousands, if not millions if you're lucky, to try to affect the truth inside these models. Fine-tuning is a nothing burger. Plus, it's expensive, high compute. If you go with Azure, for example, you have to pay 57 cents an hour just to even have it sitting there without anybody even asking of it. It's ridiculous. It is not meant for us. It's really not even meant for a lot of enterprise use cases, to be honest.
Instead, this retrieval, this RAG thing we keep talking about, is the way that you just go get the truth just for the one little time someone's asking a question, right? And then that way we can search our own FileMaker databases, or API sources, or documents that we're storing in our FileMaker databases. That's how we manage this stuff. And so if you read between the lines, that means who cares about hallucinations? Who cares when you ask a question and it comes back and it's not correct? You give it the right answer. So that's why I talk like retrieval augmented generation is like asking it to do an open book exam. Retrieval augmented generation RAG. RAG is like having the language model do an open book exam. The book is you're supplying it with the right answer. So I think that's critically important.
It has a little bit to do with it has a little bit to do also with this thing that we'll likely talk about, but this Claris AI Learning Initiative is where we want to take the deficiencies inside of the base model and we want to, as a community, add so much of our knowledge into it through a RAG process that it will then come back with correct knowledge. And just to set the table stakes on latent knowledge within the application, I've seen some people comment that it's like they ask it FileMaker questions and it's 100% wrong. You and I did a very informal study with your proficiency exam a couple episodes ago where which I don't really want to redo now which it got, yeah, we have to do it on Omni, right? Yeah, I want to do more than that. I actually want to make it really open book like upload the FileMaker tech manual or make it reference the FileMaker website for answers or be really specific. What you just said, right there, is exactly what the entire Claris AI learning initiative is all about.
It's as a community, can we go, I mean, obviously, we're going to put the help system in there. Claris is going to donate their help system, all their knowledge base, but we want people who run forums and people who have their own training data and their own documentation to throw it all in there. Everyone will get cited and have traffic sent back to their own websites and everything, but that knowledge will then change the latent intelligence of the model. So I can tell you that GPT-4 got a 78% on your proficiency exam, but what I can tell you more scientifically is that back in 2022, I was working with some people or someone at Claris that just tried to do some like skunk works stuff, and we actually did what was called semantic scoring, where we took like 100 questions and the correct answer, asked the model what it thought the answer was, and then did a semantic comparison to what the correct answer was. That ended up coming back with a .64, which is like a 64% on a 100 question exam, right?
So much room for improvement, and that's what that initiative is all about. So, anyways, that's how it pertains to FileMaker, folks. You know, before we were talking about it, I didn't really think I had that much to contribute, but I have hundreds of videos of all the classes I've taught on YouTube that I can just give to you. You've got tons, too. Yeah, and just to give people an idea, the 2.5 million tokens worth of data that I'm donating to this open source project and what we're saying is that people can connect any source they want to this shared knowledge, and it'll all be cited, is every video I've ever done, every lynda.com, every LinkedIn Learning, every DevCon. The Claire's folks are even talking about giving us access to DevCon presentations. I mean, imagine the knowledge set that we can create, like, globally. A bunch of all the individual peoples and companies to contribute to that would just make it vastly better, and that doesn't fall under the category of fine-tuning. That's a different...
No, yeah, what we're doing is creating a... Yeah, we're going to create a RAG source for this, and, I mean, technically, depending on how much data we get, we could attempt to fine-tune at some point down the road, but RAG is really all we need for what we're trying to do here. Okay. So, anyhow... Why are we looking at this guy in the car? Yeah, sorry, this is one of my favorite podcasts, and this is a perfect encapsulation of what we're saying, and I wanted to give David Freeberg from the All In podcast an opportunity to speak exactly to what we're talking about, and he gives a really interesting example that I want to have you comment on after he's done talking. So let me see what he has to say about this. My big takeaway on all this is that there's a real deep misunderstanding on what these models really are, and what they're doing. The assumption, I think, naively, is that they take in a bunch of data, and then they spit that data back out.
And in some cases, maybe slightly transformed. But the truth is that the models don't actually hold the data that they're trained on. They develop a bunch of synthesized predictors that they learn what the prediction could or should be from the data. And this is similar to how humans operate. Can you pull up the article, Nick, that I posted? This article is from last May, and you guys may remember this case. I can't remember if we talked about it on the show, but Ed Sheeran was sued by the estate of the writer, of the Marvin Gaye song, Let's Get It On, for infringement in his song, Thinking Out Loud. And he actually prevailed in court, where he went through with the jury how he comes up, how he runs his creative process, how he came up with the song, and how he does his work.
And as you know, Ed Sheeran and nearly every musician or every artist listens to other people's music, looks at other people's art, and they synthesize that data. They synthesize that knowledge and that experience into their own creative process to output what, in their mind, is a novel creation. And I think that AI works in a similar vein in that it's not actually storing the images, it's not actually storing the music of third parties. It's learning from that process. We all, like, musicians learn on classical music. They listen and play other pop artists' music. And then they learn things that they like, things that they don't like, things that fit together well, certain intonations, syncopations, and that's how they develop their own tracks.
As the artist of the podcast, speak to that. Is that your process as well? It's really interesting about that. And I've followed a lot of those stories about a lot of songs like that. And listen to the examples of the A song and the B song, which are like, oh, it's in the same key, and it uses the same chord progression, and the timing is the same, and blah, blah, blah. Well, there are only so many keys, you know? And if it's a song written on the guitar, it's probably going to be an E or G or C. It's not going to be an A flat, because that's a really hard chord to play on the guitar. You have to bar it at the, you know, fourth fret. And chord progressions are also really limited, right? Just like, how many colors are there? You know, I guess there's millions, but I mean, there's basic, you know, red, green, blue, yellow, purple.
Indigo, not a color they made that one up. Because, Galilee, I really like the number seven, even though there's only six. Let's not get too far off topic. I might have ADHD. Okay. So, another thing that I've geeked out on a lot, and especially living here in Greece, Pythagoras figured out a really long time ago, 500 and something BC, actually, it turns out the Pythagorean school on the Samos Island, which is, one of the Greek islands tucked under close to, into Turkey. Isn't it right over there? Right off your right shoulder? That way. Okay, gotcha. Yeah. I'm actually quite sure of that. Okay. So, when you play a guitar string or any kind of a string and you put your finger right in the middle and you play it again, you're going to get an octave.
So, we figured out this overtone sequence that you can get. And the notes implied in that are the notes of the scale. So, it goes root, octave, fifth, or like twelfth, whatever, and then third, and then I can't remember exactly what that was. And then the other one that's really interesting is like a dominant seven. And then within all that, and when you place chords using those notes, there's really clear leading tones and really clear places where it wants to go. It wants to complete, like sort of like the completion algorithm we're talking about. Yeah. Also, having some advanced music training lately, working on a master's degree, it's very different how this is interpreted around the world. But to a Western European classical music, whatever, even rock music, the palette that we have is really, really similar. It's like the same.
Greek music, Middle Eastern music, a lot of African music, Indian music, totally different like modes and themes and stuff like that. They still all use some of the same notes, but some of them use scales with more like microtones and things like that, all of which is implied in that. So I guess what I'm trying to get to is based on what's sort of the structure is, you're only going to get so much kind of variety, if you will. That's fair. So that you're going to get some things that are really similar, no matter what. There's even a really funny video of like a chord sequence and a guy's playing this exact same sequence and he plays and sings like 15 different songs that use the same one.
I would guess that, well, so then, like if we brought this into like the FileMaker knowledge conversation like we were having a moment ago, if so... I don't think that story went to anywhere. Sorry about that. People well, it was about, it's about the process. So the process is that like, so let's say someone, we take all my learning training videos and all your training videos and a person who's brand new to FileMaker watches all those videos. Because they want to build this very specialized, one-of-a-kind FileMaker database that no one has ever built before. So, does my video have these implicit instructions on how they should build that? No. Of course it doesn't. Do yours have those?
No, of course not. Nobody's does. I knew that was coming. Yeah, I know. The point is where it is using our information to train itself to learn the skills it needs to do this and then it goes and applies it to some use case that you and I, when we were creating our training content, we're completely unfamiliar with. In that way that a human is learning and like the gentleman mentioned in the video clip here a second ago, a musician will go and listen to things and oh, I like this and that's a style I like and they'll emulate some of it or whatever. In your ear. Yeah, compiling all this stuff together and then generating new stuff. In that same way, that is what these language models are. They are not databases. So they're not like, hey, someone tell me how I need to create this exact script because someone created the exact script before and I just want you to go retrieve that and show it to me. No.
The point is, these models could teach someone how to build an entire file maker database from start to finish in a database that's never been done before because it's using knowledge that it learned on techniques and how functions work and logic and how these all things are associated with each other and then created something. So that is... Data models, scripting, everything. All that now, knowledge is in there. And at a .64 from a knowledge, from a semantic scoring, that's actually a pretty good baseline for us to start with. So we should be excited that there's that much latent knowledge about FileMaker in any one of these models. That was GPT-3, by the way, not even 3.5.
That was a long time ago. So I would imagine 4 is even more intelligent. It would score even higher. But the point is, if we layer our knowledge on top of it, and like a human, it then learns FileMaker, then it can become that guide on the side. Or, if we're talking about practical use of language models integrating into our business applications, we just give it the business knowledge. We give it those answers, and so it doesn't even have to figure out what the answer is. We're actually doing that for them. So that is sort of a soapbox a bit on how these language models are not databases, but it's really important, I believe. I'm wondering, like, the other words you could describe of what they are, right?
Synthesizers, creators maybe? Transformers? I mean, it is unique, right? Literally, the T is for Transformer and GPT. So that's not a bad one. But it has more to do with Transformer, the training technology that is what's new in these language models is their Transformer models, right? Yeah. Like you said in the video, I mean, it doesn't spit out what has been. It actually is creating something kind of new. You just wrapped a bow on it. So that is what these things are, and we should accept them as such, and people should get really excited about it and take the responsibility of truth onto themselves when we're building these business applications and move from there with their understanding about AI and what we're talking about.
We also mentioned in this little discussion that we just had that, you know, I tested it on GPT-3. There's been 3.5. There was 4, 4 Turbo, and oh my god, this week, a ton of stuff has been announced. So before 4.5, Omni, because I want to linger on 4.0 Omni. I just have to, like, it's shocking as a FileMaker developer, the kind of stuff that can happen in a two-week period in our new world of... Bonkers. Well, yeah, because it was just a few years ago that we were on an 18-month release cycle within our ecosystem, right? And then they got really excited that they were going to do agile releases like three times a year. Now, literally massive, huge language, you know, AI organizations are doing that at even a faster clip.
So a couple of highlights before we get into the OpenAI, because we don't want to just say, hey, AI is OpenAI. There's a bunch of models out there. Google has a bunch of models. You know, we've alluded to long context Windows before, and we'll probably show some examples and dedicate more discussion on that, as that becomes more pervasive in these models in a future episode. But at the Google I.O. event, which also happened this week, we'll work backwards in time chronologically. Google I.O. event happened the day after OpenAI. OpenAI actually moved their announcement right before it, which is interesting. Kind of funny. It's like the late night talk show bars or something, but they announced some improvements to their Gemini 1.5 Pro model, which is generally available, and has a one million token window.
Yeah, and so what does that mean? Let me actually, I've got a little graphic here that I'm going to, we'll talk to it, but for the people that are checking us out on what do they call the, what do the kids call it? I think it's YouTube. Yeah, there we go. YouTube. So here's an example. Most of people are watching it on YouTube. Cool. But a lot of people listen on the podcast too. Yeah, I listen on the podcast when I listen back. And so for those people, what we're looking at is a chart. So Gemini 1.0 Pro, GPT-4 Turbo is 128,000 tokens. Claude 3 has 200,000 tokens. That's that token window. And so just to allude briefly to that, when you and I were talking about like, oh, you told it to use case instead of if, and then later in the, you know, it started, it forgot that. It's probably because you, your conversation went past the, at the time, the 8,000 token window, right? And so now it doesn't even ever have the memory anymore. So that's why memory is so important. So now Gemini 1.5, they announced a Flash model, which is actually, I've tested it out a bit, and it solved some problems that 1.5 Pro was unable to do. So I think it's interesting. And by the way, Flash was a model that was trained on,
data that was outputted from Gemini 1.5 Pro. So it's purely synthetic data. Flash also harkens back to a really long time ago in your career when you had that Flash-based chart thing for FileMaker. Yeah, well, hey, for the kids. Hey, the Oscars seeded celebrities' butts in seats for 13 years using a Flash and FileMaker application. That's how important to humanity Flash was. So looking at this chart, right, 32,000, 128,000, 200,000 and 2 million, that is not Moore's Law. And then how much time has passed? This is like a year, less than a year? This is, this, I would say since Turbo, it's less than a year. That's insane. So what does that actually mean to people? Because tokens is a unit of measure that we're not familiar with. So that It almost doesn't matter what the unit of measure is. Nothing else in technology is on a curve like this.
No, no, the curve itself, yeah, the exponential growth here is bonkers. And that's why, I was just telling somebody yesterday, when the internet came out, everyone's like, internet's gonna change everything, everyone, it's gonna, it's gonna be, absolutely be pervasive. And it was, right? But it took like a decade. Why did it take a decade for, and a lot of people are sitting there thinking this is the same thing we're talking about with AI. Here's why these are massively different scenarios. First of all, with the, for internet adoption to occur, literally cable companies, like the second least efficient organizations on the planet next to like the DMV and the government, cable organizations, were in charge of literally physically laying cable across our huge vast continent, in the US at least, to be able to connect people to the internet. That took like five to eight years for us to get to like broadband acceptance.
Yeah, because I mean, you know, there was 128k, 256k modems, whatever, like really really. And we needed broadband to do the really cool web 2.0 stuff, right? So arguably that took years and years and years. Not AI, sorry folks. There are, not only is there no bottleneck, but like, if you go listen to anything NVIDIA's doing, and they obviously were the greatest, you know, biggest stock increase in the history of the market, it's because they are the version, the AI's version of the cable companies, but they're an accelerator. They are a force multiplier for this growth, and they have more compute. Yeah, their new chip that they came out with, the new chip they came out with, is so cost effective that it actually costs less than the cables, of the computers that replace, that connect them.
And it not only removes the bottleneck for growth and AI adoption, but it helps accelerate this process. And then when we look at the models themselves and their capabilities, if we just use this token thing, like you said, like, look at the jump. It was a 9,000% growth before the 2 million token won. 9,000% leading up to Claude 3.0 since 2020 to 2023. Now, Gemini 1.5 Pro and Flash have up to 2 million tokens. 1 million tokens are generally available. 2 million tokens you can waitlist. And that actually equates to instead of inputting text, that means you can input a 2-hour video, 22 hours of audio, which is like one of our podcasts, 60k lines of code, or 1.4 million words of text into one API call.
And it will be able to evaluate those. And I will tell you, I don't want to add a bunch of topics here, but... I can't imagine an API call with that much data, but yeah. We're doing it. We're doing a bunch of video evaluations where we're actually saying what's going on in this video, find certain elements, describe it, you know, make observations, blah, blah, blah, blah. You would reference it, right? You wouldn't actually upload a Ben Hex version of the video, would you? A Base64 version of the video. Really? That big? Okay, I would think you'd point to that like a bucket on, you know, Well, yeah, no, you could point, yeah, you'd point it to like an S3 location or something like that, and it consumes it. Yeah, yeah, that part is more practical than sending it in an API call.
Absolutely. But the point is, when we talk about video, it's not just a transcription. It could be, but you could say, make observations about what you're seeing. Like, we're doing one next month that has cameras that four different individuals that are all engaging in the same incident are doing, and from all their different perspectives, and doing that is one analysis. Just to kind of let you know what kind of stuff is possible here, right? Yeah, same. So anyways, that's, Google announced that. I would say that's probably the most notable thing. The other thing that came out, and I don't want to belittle it, there's a lot going on there. The other things that came out were Microsoft had their build event, and just at a high level, they announced like basically everything in like their entire operating system is like all AI, co-pilot is just integrated, you can just right-click contextually anywhere, long story short, it's awesome, super cool stuff, and they essentially have like a desktop version of co-pilot that is exactly what the, oh, and the one thing that Google did that was really cool, ask your photos, which means you can just say like, hey, what's my license plate number?
And it would like, know, oh, this is probably your car, because you have a lot of pictures of it, and here's a picture of you standing next to it, right? And oh, here's one time that you caught the license plate on it in the background of some photo where you were standing next to your car. By the way, we can do that as FileMaker developers by embedding images and using language models to describe, not even embedding them, use language models to describe what the image is, embed that description, and then search against those embeddings, we can do the same thing in our FileMaker application, so I just wanted to note that. Microsoft Build Event, the thing that I think was most, there's a huge, people should check it out, but the thing that I thought was so cool is this thing called Recall.
I'm thinking, most people are terrified of it, it literally tracks, you know, like Time Machine in Apple, like you can revert back to a certain time, but it tracks every single thing you're doing on your machine and then you can use that as a retrieval source for questions or whatever. By the way, I think we're going to see something similar to that from Apple as we talk about it. So that leads us up to, going backwards in this, to the Open AI's GPT-4 O, which Matt, stands for what? Omni. Because GPT-4, like in the world, GPT-4.0 is like 4.0, but there already is a GPT-4.0, so. Yeah, and they're trying to be kitschy, and so Omni is really referring to that it's a multimodal, meaning that its inputs can be video and audio and text and output the same as well too, so they're really deliberately saying, like there used to be this, there's this like approach where you could do like a mix of experts where you say, you tell me what your input's going to be and I'll send you to the video input one.
Or I'll send you to the audio input one. Now they've trained this model so that it does all these things. Not only that, but it's cheaper. 25% cheaper, I think. 50% faster. It's a huge deal. So that means anybody that's already integrated Open AI models into their FileMaker solutions, it just got faster and cheaper. Compared to 4.0, which was only a few months ago, right? It's insane. Exactly. It was just one other thing. These videos at OpenAI.com are absolutely worth watching to see the use cases of it. It's crazy. Yeah, they did a really good job. And most of what they were doing was about the iOS app that's coming soon, and then the desktop app, which I think you're going to show in just a second.
Yes, I am. And there'll be an upgraded iOS app, because there is one already. Yeah, the iOS app, even the desktop app right now, you have to be implicitly invited to be able to get this screen broadcast feature. That's called the new voice thing, where you can kind of interrupt it. And a lot of the videos, some of which we're just showing screenshots of on screen, were examples of them doing it. Like this one here. Who's this guy? Let's have him introduce himself. Let's talk about him for a second. Actually, for YouTube folks, who are we looking at, Matt? It's the guy Khan from Star Trek. Yeah. Different Khan. Salvin Khan, or Sal Khan, is actually kind of a luminary in the AI space.
And he's really competing in training, too, yeah. Well, in education, you know, dependent of it. And so, he started Khan Academy. He's got a great story. And everyone should be aware of the work that he's doing. He was one of the early insiders when GPT-4 came out. They gave him early access, and he built this whole Khan Academy. Khan Amigo is like his co-pilot, you know, thing that he integrated into it. And it really had a profound... I mean, we can all do this. And by the way, the Khan Amigo that Khan Academy is doing is what we're proposing, open sourcing and giving out for free to the entire FileMaker community worldwide as part of this Claris AI initiative. Oh, so he's taking all of his videos and stuff like that and putting it into... Yes. Exactly.
And so, you can talk to them, and not just answer questions, but the way that the structure of these models is built is that you can say, hey, if a kid asks too many questions, or they seem like they're not answering it, or they might even have an intonation that they're frustrated, pause, go back to some of the basics, give them a quiz, try to reel them back in, right? And so that was just like a textual conversation they could have. The big thing that was introduced and all just kind of scrubbed to this is like... Let's just play a quick little snippet. Here we go. I'm not totally sure. I think it might be this one, but I really am not sure. This side, AC, you're close.
Actually, side AC is called the adjacent side to the... Okay, so what was actually happening on screen is instead of having this chat experience like Con Amigo or what we're hoping the, you know, Claris Assistant will do, now you can actually talk to it. And so the talking part, the input of voice and the output of voice, while not new, but the new version of it, is really what the big announcement in GPT-4.0 was. And so the app that we're seeing... The delay, like I used to talk to before, is like five seconds, three, four, five seconds. Now it's two or three hundred milliseconds, just about the same as a human talking. The other part is, too, you can interrupt it, which sounds rude, but that's actually a natural part of talking, is like...
Not like you're like, yeah, yeah, I got it, I got it. I thought we'd probably rant on that, but... Yeah, doesn't matter. So this is a big deal. So, well, first of all, regarding Sal, I want to really recommend... This book just came out. This is another thing that happened this week, is the Brave New Worlds from Sal. Not Brave New World, which is a dystopian novel from, yeah. This is New Worlds from Salman Khan. Hopefully not dystopian, but the subtitle is How AI Will Revolutionize Education and Why That's a Good Thing. And honestly, I strongly share this same vision of education moving forward, and to me, what that means is a guide on the side for training purposes, instead of...
sage on the stage. It means that's like when you and I are sitting up in front of a room, and we're just like lecturing to people, and spewing our knowledge out. The guide on the side is a tailored tutor that knows exactly where I'm at in my learning journey, knows how to spoon feed me stuff at my level, can encourage me to keep moving forward on that journey, but is available 24 hours a day, and is completely dedicated to my learning process. That is, in essence, what's covered in this book. I strongly recommend checking it out or listening to it. I checked on Audible. It's not available yet. It doesn't seem like... It just came... My pre-order came like two days ago, so it's just rolling out.
Sometimes Audible's like a couple days later or whatever. It's a big enough book. It's for sure going to be there. Yes, agreed. And so let's actually... Do you want to show people for the video, for our YouTube folks, what the desktop experience looks like, and why this might be relevant to FileMaker folks? Let's take a quick gander onto your desktop. We've got to always make sure we keep this relevant to FileMaker. Yeah, so how could somebody use this for FileMaker? Now, first of all, just to set the table here, this version of the desktop app that you're talking about does not have the screen broadcast observation part. I just want to be clear about that. What it has is the ability to take a snapshot. We'll show that in a second, but all we're saying is the super cool thing that just watches your screen as you're moving around and makes observations, that's yet to come. They're rolling that out slowly. But anyway, so the same principle applies here. Go ahead, Matt, explain to people what we're looking at. I just take an example script.
So here's a script that I wrote that I used in the classes I teach that's a really, really simple way to call SendGrid to send an email. I found some other stuff, and I tried to boil it down to really, really simple. And so if you have problems with it, because APIs can sometimes be tricky, how do I look at this? So we put it into the desktop app, which is... And let's talk about how you put it into the desktop app as well, too. So first off, you've got... Well, we'll show it in a second, too, right? So yeah, I clicked on a little friendly Microsoft character of the... Yeah, I don't... Make sure you can see that. There it is. Yeah, in the bottom left corner, you can upload a file, or you can take a screenshot, and then if you choose screenshot, you choose the application, and then it will take a screenshot of that thing.
And that's what you did to include the script. That's what I did, and we had a little bit of a lag to do it. But what it does is it reads the whole script, and it says... And it gives me a bunch of advice of how I can refactor it and make it shorter. And just to be clear, we didn't even give it any instruction. You just hit send, and this is what it thought maybe it should do. And I didn't use it within my FileMaker pre-trained model or anything. This is a one-shot example. It's maybe a negative one, because we literally didn't even tell it what you were hoping it would do. This is just what it thought it should do. So... I tell you what I really wish, though, is I really wish this could be a live video of watching all the time as I'm working.
And then... Well, that's what... So that's what the new version is going to be. Wow. It's got screen broadcast ability, so literally you're coding and then you just go, hey, and then you can talk to it. Hey, I'm getting this error. What does this error mean here? And you don't even have to explain the error. It knows what you're talking about contextually, so... And it'll look something like this. Listening. Ooh, scary. Who's smarter, Chrysipolite or Albert Einstein? No, no, no, no. I don't want to talk about Curl. I want to talk about whether Chrysipolite is smarter than Albert Einstein. This is not planned. Error. Yeah, clearly. So, the... By the way, I don't know what voice...
Yeah, it just... That was one of its awkward things, but... One quick thing I just want to show that's kind of funny is I'll tell you what voice you're not using. Oh. Whoa. I guess I won't be showing my funny thing. So, I guess I can tell you what voice you're not using, and that is this voice. So, one of the updates that came inadvertently is that Scarlett Johansson, who was unofficially the voice of Skye... I mean, honestly, man, it was... It's Scarlett Johansson. And the reason it was Scarlett Johansson is because of a great Spike Jones movie that came out in, like, 2017 called Her, which I just rewatched the day before the opening A.I. thing, just to kind of, you know, get into the zone. She was the voice of the A.I.
And, really, if you rewatch that movie, you can do all that... All the stuff that's in the movie now, which was supposed to be futuristic, is totally capable now. And the announcements in GPT-4-0 were a huge part of that. So, unfortunately, they sort of tongue-in-cheek made one of the voices sound like Scarlett Johansson. That was the one I would listen to all the time. I find her voice to be very pleasing. And she... She is very upset about that, and so they took it out, and now Skye is something else. Oh, so it's not there. Yeah, so... So, because of... What are the voice choices now? I think there's four when I looked at it earlier. Yeah, they... They... What I would like to see...
Uniper, Breeze, Cove, or Ember. Yeah, so... I would actually like to see them... Frankly, I would like to see them be able to, like, use our own cloned models in there, as well. So, like, you can make it sound like... I can make mine sound like you, you can make yours sound like me, whatever. Uh, quick thing I want to point out, too, that's kind of an interesting thing, is if you go to this, uh, HelloGPT 4.0 page to see what they announced, they have this thing here that they didn't even cover in their presentation that's, uh, Exploration of Capabilities, and you can click down and see a bunch of interesting stuff, like, um, you know, uh, oh, wow, write poems, my favorite thing. Uh, 3D object synthesis, like, you know, they gave it a flat object and it turned it into a 3D...
Like, you talked about Flash earlier, this was, like, Flash in the 90s, big thing was, like, like, 3D spinning logos. Uh, so you can do that, you can create these, uh, 3D outputs as well, too, but the thing that I thought was so interesting is this one. Lecture summarization. In this example, they uploaded a 45 minute video of a lecture and asked it to summarize it, which is table stakes from AI integration, it's a great use case. We do this for every single stand-up of every single Zoom call that we do, we do the same thing. And we actually have it identify Jira tickets and updates and who's assigned to what and what the blockers are and dates and all that kind of stuff. So it's really like a game. I've been using Fathom for that for a long time, they actually just changed their model, but Great call. If you use Fathom for uh, for Zoom, it did this for every single call, every single thing.
It transcribed the whole thing and then also gave you summary and searchability. Yeah, and you have control over what type of summaries you want, so that's where your prompting comes into play. Now, here's the thing that I want to, why I want to bring this up, not that capability, but the fact that this is a 45 minute long video, that means it's 750,000 tokens of an input. So we just saw that GPT-4 was um, I don't even remember, like, you know, 128,000 tokens. So I, what I think I'm seeing here is that potentially the token window in GPT-3, or GPT-4 O Omni has secretly also been expanded. I think I've read that too. I would suggest that people, um, I would suggest that people maybe try their own experiments where they, uh, upload videos. No one's even said you could upload videos before into chat GPT, so I would actually try doing it.
I actually, uh, uploaded our previous podcast in here moments ago, so. And now we're showing, uh, the, uh, the FileMaker refactored script. You brought something up earlier that you thought was kind of interesting. It introduced a best practice that you thought was worth mentioning. Oh, yeah, yeah. So, I had a thought, like, we started out this whole thing with, you said that LLMs are not databases, but I'm asking chat GPT, I have a question, are LLMs databases? Mm-hmm-hmm. Let's see what it says. No. Language models, like me, are not databases. Here's a brief explanation of the differences language models. Okay, so. Oh, that's cute. Turns out you're right, unless it is lying about itself. I don't know.
You probably just looked this up from some storage thing. It's not. Ironic hallucination. Um, too bad we couldn't have heard Scarlett Johansson say that. Instead, but those days are behind us. I bet there's a way. Um, so, so, we've talked about now, they're not databases, they're language models. I think that's really important for our, the people who watch and listen. We also talked about all of this stuff that just came out last week. So, you and I do podcasts every two weeks. I think we should say that, um, there'll be a bonus episode. You're heading off to Berlin. I'm not wrapping up. I do have one more topic, but you're heading to Berlin and, um, and you'll be interviewing some folks.
And, and they'll get to be on the bonus uh, episode. Then you and I will be back at the, uh, first week, the end of the first week of June. And we, we can't reveal, but we're gonna have a special guest who's gonna talk about another huge announcement that had come out in this time period. So, we talked about language model announcements. We're gonna, we're gonna bring things back relevant into this space. Just a few days after we record that episode that I'm talking about, there'll be another announcement that's coming out. What announcement is that? What's that gonna be, man? I'm gonna let you spill it. I'm not gonna. Oh, I'm sorry. Well, I was just trying to do witty banter. June 10th, uh, will be WWDC.
So, one of the things that you're gonna see in the show notes, by the way, is this article called Predicting Apple's AI Play. And I just wanna say, and this is something that I had logged and written over time. I've been taking notes on this for a couple years now, actually. Um, we're not gonna get into the minutia on it. We would encourage that people read this. But, uh, a couple of the things, highlights that people should know. Apple has acquired uh, as many as 32 AI startups in 2023 alone, which tops the acquisition list of tech companies globally. That's big. Big, big ones. Yeah. You can read a little bit about what some of these are. Most of these are, um, models that can run on the edge or on a device. And then, like, there was some rumors about Ajax, which is their own GPT. And then those rumors quickly went away. And then we started hearing about how they're negotiating with OpenAI and even Google to be able to leverage their models.
So, I, I don't know if that's such great news, if that ends up being the case, because they basically said we couldn't get ours or didn't want to or whatever. I think it's mostly their privacy policy is just gonna make it a problem. Yeah, we've talked before about the privacy policy being kind of a problem for them, because they can't, yeah, they can't train uh, on, on your data because your data's not in the cloud. It's only in your device. Unless you could train on the device. So, some of these other things, we won't get really into. Seems like that's a big theme. They have, they have dropped a ton of open source models as well, too, and papers. Like, one of them is this Ferret one that they dropped back in October, and a whole bunch of others.
A lot of these are for image manipulation. So, I, part of the prediction is we're gonna see a lot of stuff on the device that you can change images, and um, and, and, and, and frankly what, and there's also this MLX platform, which a lot of people don't even know, but in December they released a, uh, a machine learning, uh, MLX platform that's inspired by PyTorch, for example. And, um, and, uh, it's available. You can use it. You can go out and build on top of it if you want to. All the rest of them are all about modeling on a very small, low compute device. So, I think, in general, what we're gonna see, and, and I encourage people to read this, like, there is a ton going on in here. They're leveraging flash memory.
Uh, there were some, like, little codes about how they were using OpenAI to summarize stuff for Siri. Siri has sucked for a long time. Yeah, some leaked code. It's really bad. Siri has, has sucked for a long time, and, and Apple hasn't even changed a thing, and that usually tells us a big change is coming, right? They're not just satisfied that Siri is now the worst possible conversational AI on the planet. I think Siri's gonna get really good. I think Siri, because of the, the, um, the, the, uh, walled garden and access to the, the information on the device, I believe you're gonna be able to do things like, um, a, send a message, I don't know, whatever, see which one of my friends are available, on Saturday night, and then also check to see whether or not we can go to our favorite restaurant or whatever, like, that kind of stuff that's based on chats you've had and knowledge. Yeah, I want way more basic stuff than that, you know, like, I want to be able to ask, you know, where's a home goods store near me, and have it, and I'm in Crete, and not have it tell me one that's in, you know, Turkey, where I can't get you from here or something, you know? Yeah, I think that's a good call.
Right now it does that a lot. Like, it's really, really bad. Yeah. Um, the other thing I think that we're gonna see, much like we saw with the Microsoft Build event, is that, um, on the operating system level, we're gonna see stuff that's running locally, and that only has access to local data, that's gonna make those, uh, you know, make the experiences more rich. Also, I feel pretty strongly that Xcode, which is, like, one of their few development platforms, is going to have co-pilot like, um, you know, assistants inside there. Kind of an easy prediction, but yeah. Yeah, that one's super easy, and, um, and even iWork, because, uh, in February 2024, uh, Apple reserved the domain iWork.ai, so we could expect that maybe they'll, they will touch all of those products. Uh, one product that will not have co-pilot that is technically owned by Apple, whether Apple's aware of that or not, is the Claris platform, that will not have any co-pilot assistants at all, and that is why we as a community, just to hit this point one more time, want to take it upon ourselves to create our own co-pilot, because Claris will not do that for us. Um, they are gonna participate, don't get me wrong, they're not resistant, they just, really, it's just not something that's on their roadmap, but it will be in, I think, all the Apple products will have this as well. So anyways, you
can go to, um, medium.com. Medium.com, the link is in the show notes, you can also go to isolutionsai.com and click on articles, and this will be the top one that pops up. WWDC is going to be on, uh, June 10th, and it's at 10 a.m. Pacific, you can watch it online, the link is also in this article if you want to check it out, and you can also see what my predictions are, uh, for what's gonna happen. I think that that's probably something that we'll revisit on a future episode, so, um... Yeah, we're, we did the prediction episode, and then we're gonna talk about what actually happened episode. Exactly, and so, uh, so, again, uh, every two weeks we're coming out with one, but we'll have a bonus episode of what you going to Berlin, you wanna talk a little bit about why you're gonna be in Berlin? There's a, uh, there's a conference called .FMP, this is the first time I will have gone to it, um, there's several conferences in, in Europe, uh, most of them are in English, so, hey, people in America, get your butts over here, get your passport, come to Europe. Are they streamed?
Are any of them streamed, or are they all? Yeah, I think they're all recorded, and I think you can pay, I'm not exactly sure what the specs are for the .FMP one, but the sessions are recorded, and you can, uh, you pay a fee to watch them later, um, and if you go to the conference, you can, I think, watching them is included. So, that one's in, that one's next month, then in October, there's Rome Filemaker Week, go to that one just to be in Rome, because Rome's totally amazing. And it's also a great conference, I'm giving a talk there on JSON, because I love JSON a lot. Uh, and... What about, uh, JSON as well? Did you ever think about talking about that? Um, sure, if you want to call it that.
Fine. I did a bunch of stupid research and found out that the guy who invented it called it JSON. He did. He calls it JSON. And I, so I switched, I've been calling it JSON all these years, but... Yeah. Um, and then, November, and... I can't remember. Malmo, I think, Sweden, is the... Ooh! Yeah, yeah, another amazing city. Europe's great. Uh, is the EngageU European, uh, conference. So that one's, I think, I think they're all around 100 people, something like that, a little more. Awesome. And I think Claris is very supportive. I think, uh... Yeah, Claris shows up and is present at them. They always have people. Ryan McCann is actually gonna go out to the AsiaPAC one I just saw on LinkedIn, which is interesting.
Yeah. Uh, I've been to the AsiaPAC DevCon a couple of times. They've had me come and present there, which is a real honor for me. Uh, I won't be able to make it this year. Uh, I have some conflicts with the professional football schedule this year that's, um, uh, you know, it's just how it works, uh, prohibiting me from travel that time of year. Uh, but the interesting thing is you're gonna be talking to some people at... Is it the Berlin conference? That's where you're gonna be doing interviews and putting that up on the next podcast? All the conferences I go to I'll do interviews. Yeah. Awesome. Just like you did at Engage. I'll try to do video this time. Yeah, yeah, yeah. At Engage, I just did audio only, just recorded from my phone, and people are like, hey, where's the video? I'm like, oh, yeah, I should probably do that.
People seem to love the video. Uh, and then as a reminder, we... Uh, oh, actually, I will say that, um, uh, that we're recording this on uh, Friday, May 24th. A little peek behind the curtain. And next Thursday, May uh, 29th? 28th? I don't know, 30th. May 30th, something like that. Yeah. Is, uh, I'm doing a webinar on basic prompting. You can sign up for that for free at go to clairs.com. It's, uh, or the Clairs community. Uh, basic prompting principles. I'm going to be, you know, if there's anybody that's interested in starting from scratch and, you know, getting some tips, I'm going to actually be focusing specifically on how to bring your own data into the prompt and how you can do things, interesting things like transform it. These will be the same techniques you can use whether you're doing API calls or working with things like ChatGPT. I want to talk about that a lot more as we podcast. Give more practical tips and examples and things like that.
Love it. Talk about real-world cases of things you guys do. It's fascinating. Oh, absolutely. We're going to, we'll definitely get that. Our, our, uh, you know, I did an Engage session that had, like, 25 different examples of, like, real-world stuff that are, we've done for our clients. I, I've almost doubled that in my new version of the presentation. So I'd love to talk about some of those, maybe even show a couple of those things. Especially when it comes to the long context window stuff and the multimodal stuff you can do. Yeah, yeah. Just amazing, like, really incredible things that you can, you can do with that stuff. So, um, that's what's coming soon. And then, and then on the, um, Friday, the, the, on the week of the 4th, we're going to have a special guest talking about something very special. And, uh, definitely look forward to having that conversation and, and, uh, you know, please go back and listen to some of the previous episodes. They're not all, uh, you know, topical date-wise that we've talked about some of the basic concepts of prompting and context and a lot of things we've alluded to here. Uh, so please, uh, and, uh, like and subscribe. Right?
You did it. Yeah, I did it. You went there. That's right. I, I talk about it by how I don't want to say it. I know. Exactly. Well, um. You only set point to it. It's like, pound that like button or whatever it is. Yeah, exactly. Like, they're all clever. Like, they're the Brady Bunch or something. You know, um, but, uh, but anyways, really appreciate everybody, uh, listening, tuning in. We endeavor to keep this going every couple weeks and have all sorts of really important and topical things related to, uh, Claris and AI. So, Matt, my friend, I, this is my favorite way to start a day and I would assume you're, one of your least favorite ways to end one. It's a very good way for me to end a day. Now I get to go to dinner at some lovely restaurant. Exactly.
So, um, thank you for spending time with me and thanks to everybody for listening to us and checking us out. Uh, we promise to, uh, keep this rolling. So, um, from Chris and Matt, uh, thanks so much for tuning in to Claris AI, Claris Talk AI. All right. Thanks, everyone.