navarre.ai

ClarisTalk AI · the FileMaker AI podcast

Semantic Search is Native in FileMaker 2024! Special Guest: Ronnie Rios

June 7, 2024 · 95 min · Matt Navarre and Cris Ippolite

Jump to transcript ↓

Show Notes: New FileMaker 2024 AI release: https://support.claris.com/s/answerview?anum=000041889&language=en_US Links: Claris Community Live: Learn about AI prompting basics: https://www.youtube.com/watch?v=AQlOu-BIs5U See CRIS at Summerfest-Tech “A.I Deployed: Real World Examples” on WED June 26th in Milwaukee: https://www.summerfest-tech.com/summerfest-tech-2024 Join the conversation on the Cla

Apple PodcastsSpotifyYouTubeAll episodes

Transcript

Transcribed by machine (whisper, large model), 18,460 words. Names and product terms are sometimes misheard; the audio is the record.

Claris Talk AI. It's another episode of Claris Talk AI. Hello, Chris. Hello. We have a special guest today. It's Ronnie Rios from Claris. Welcome. Hello, everyone. Hey, Ronnie. I'm really excited to be here, man. This is really great. I'm really glad you're here. Yeah, me too. I get to talk about my two favorite subjects, FileMaker and AI. This is going to be awesome. It's going to be a lot of fun. For sure. Thanks so much for inviting me. Anything big happen three days ago? I don't know. I can't think of anything. New episodes of my favorite show? I don't know. Let's not keep people in suspense. Final Maker 2024. I think I added a 20 in there. Shipped three days ago. Exciting.

Bunch of new AI features. I'm here in Berlin at the .FMP conference seeing a bunch of cool sessions about it. But I would really love it, Roni, if you could take over and show us some cool stuff. Talk about what you've got to talk about with a new version. and i think the reason ronnie's here and ronnie you want to talk a little bit about your role in bringing these things to market i think will kind of frame things up nicely for everybody sure that'd be fantastic um so chris knows chris knows i've been working with uh with ai it's been kind of my little passion on the side uh for quite some time and we've been you know everything from from coramil to just about anything that will uh come my way um but you know obviously with generative ai a couple of years ago i started uh pulling around with some of the things that it really enables enables you to do so i've been uh playing around and trying really to figure out how to bring it into filemaker what's the best way of enabling bringing value to the end user right as a developer bringing generative ai and some of the things i've been playing around with for quite some time i was able to show off some of those things uh at engage to engage yeah that's You did actually an excellent job. And actually, you were representing Claris there as a product manager, if I'm not mistaken, on the AI stuff. I know your role was influential. I just don't know what they call that over at Claris these days. And you did the hit of the Claris Engage conference. You were at the time, though, not talking about FileMaker 2024 stuff, were you? You were doing what was at the time in market. Is that fair to say?

Yeah, yeah, that's right. So my session at Engage was really about showing how to bring AI and kind of more focusing on generative AI, because that's what everybody's really looking at right now. It's how do you bring that? How do you power features for your end users with generative AI? So you're talking about like writing poems and stuff like that and question answer machine stuff, right? Anything that's zero shot is good. Yeah, anything to zero shot, right? Yeah, Chris and I have always been joking like, well, people, yeah, these bots are great for generating poems and stuff like that. But really, what can I do for my end users? How do I bring AI powered features to my end users? How do I bring value in through AI?

And so I've been, you know, trying to do that, playing around with it for a while. And so my session at Engage was really about showing some of those use cases, some of the things that I've been playing around with, some of the things I think that people can start using, again, with the in-market version of FileMaker, what was in-market at the time, right, in FileMaker 2023, and how to enable those features using AI. So we had a whole session where we go through some of the concepts and showing some of the use cases there. And I think people liked it. I think it was a lot of fun for sure. Huge hit. Huge hit. You want to show us some stuff? What are we starting with?

Yeah. Sure, I can show something from it. Yeah, so we'll go into the Wayback Machine to look at some stuff that Ronnie was talking about, which will help frame up why the stuff that came out this week is so important and compelling. So, yeah, I think one of the features I showed about four or five different use cases and one of the ones that really kind of got people excited, because I also teased it during the keynote, was about called the smart dashboard. It's really how do you do, how do you generate charts in FileMaker using AI? You know, I was explaining during the keynote that one of the things that I almost always have had to do is build some sort of chart in every application that I've built.

There's some sort of report and usually some sort of charts, some sort of dashboard areas. And the problem with charts is that it's almost everything that we build as a developer is that it's going to be fixed in place for a while. Whatever we build, we kind of have to predict what the user is going to be asking, how they're going to use it. And that's almost impossible, especially when it comes to reports and dashboards, because they're used by decision makers. And I don't know how they're going to want to manipulate the data in order to answer whatever challenges they're having that day. And so to me, charts was just kind of like that really good use case where the idea here was to use AI to generate charts.

So behind the scenes, really what it does, it's the scripts and with the APIs, it's really generating SQL statements behind the scenes. And so you provide a description of what it is that you want. You pass that along with the schema of the entire database. And so the LLM just generates a SQL. In my particular case, there's no data exchange. None of your data gets sent to the LLM. And I'm not saying that's just a bad thing in and of itself, but it does make it a lot faster. If I don't have to pass along my entire table or the results of the SQL, it's going to make it a lot faster. Well, actually, Ronnie, double click on that a little bit. Like, I don't mean literally, but like, that's actually kind of a huge deal, especially if people are concerned about data privacy, right?

And that was one of the things I really was impressed with on your demo. So first of all, you're using SQL as your retrieval mechanism. We've talked before how you can do vector database searches, or you can even do API calls. API calls, yeah. You chose to do SQL over DAPI, for example, which we've had some conversations about, And I thought it was really interesting, your decision-making process there. And then did that also help you facilitate not actually sending the data over to the language model or were those independent? So the reason why I chose SQL and I've heard some people say, well, why did you choose SQL? Because it's not that efficient, especially in FileMaker.

But there is just one, the easy answer is almost every large language model knows SQL. SQL has been around for so long. there's so much data. Part of the training data is SQL that almost every large language model knows SQL. So, I didn't have to train it. I didn't have to write it. It's fairly straightforward to generate. And it's really, really good at it. Most models are fairly good at doing that. Some are better than the others. But it's just a way out. You could have done API. And as a matter of fact, I had Mike Wallace walk up to me at Engage and showed me how he had a large language model generating calls to DAPI, which is fantastic, right? Again, this is just one way of doing it.

It was not meant to be showing what's the right way, but showing that it is possible to do these things, which I think was important to do. Yeah, I think the thing with DAPI, because that's what we use in our intuition platform when we're communicating with FileMaker is, and you and I have talked about this before, but it's the context part of it, right? You're communicating then through an API in the same manner as you would have done with CWP or something like that in the past. through a layout to provide context into the data so rather than and then so and actually the togs might come into play as well too as far as context and related data one yeah one of the i want to see what this does i'm dying to see what's going on well okay so but the point is he's using sql here and they're the really important part of it is that there's no data exchange so you want to talk about why why that is and why that's important because i see the terms in the Claris messaging, by the way, all over the website about responsible.

And that's actually a big thing to bite off if you're going to say that your AI is responsible. So I think this is maybe one of the ways that Claris is indicating that using these tools is a responsible way to do it. So if you want to expound upon that a little. Yeah. So again, this example sends no data whatsoever. All I'm getting back from the large language model is the configuration of the chart. I'm getting the SQL itself and how the chart is going to be configured. What the data is, as far as a large-ranging model is concerned, really is irrelevant. And so I can separate the two things, right, the construction of the chart and the data, and so no data really needs to be sent over.

This gives me a great advantage, right, of obviously privacy and security, so I do not have to send over that. The only thing I'm really sending that might be considered for some people a little sensitive would be the schema, right? But that's a decision. But nothing else is being sent over. The other reason about that also is it makes the inference a lot faster. The less data I'm sending, especially the less data I'm generating, is going to make it a lot faster. And less expensive, too, because you're not taking on token costs, too. Absolutely. So just by separating those two things, it just made a huge difference. So I tried to do that, make it faster, make it private by design.

So because of those things, it's really fast. it's almost magically fast. I'll show this really quickly. This is what I showed during the session. I have this little dashboard that I created for my users. There's already a list of widgets. I'm calling them widgets. These are charts and indicators that the user can use and place them in some of these five places that I have over here. If they want to generate one, they click on on the plus sign. They choose what type of, which they want, maybe an indicator, and they type what is it they want. They really describe what it is. So in this case, you know, sales cycle length. They click on the button, and that information, right, this description, along with the schema and some other things get sent over. And what I get back is the SQL query that's going to be required to get the data that I need, and just a JSON object that describes how this data is going to be manipulated, how it's going to be presented. And so I can make changes over here. I can say, hey, this is going to be a number. And then I can do all that processing, all the formatting here locally in FileMaker. And once that's there, then the sky's the limit, what I can do with this.

All of this is all native FileMaker stuff. There's no plugins, no JavaScript. There's no crazy stuff over here. This is all FileMaker stuff. So for those that are listening, we are Looking at FileMaker charts, yes? It's all FileMaker charts. So, and just to kind of, you know, I think that's a really important point here because those are probably one of the more difficult things to configure and use inside the FileMaker UI. And you have managed to use GenAI to make that more approachable. So approachable that users who may not even work for the company that's going to be using the system that you're building this for, they get onboarded years to come. they will just be able to interact with this and get exactly what they need, right?

Like that's, I think, the really interesting part of this is how dynamic this interface is. Yeah, it's extremely dynamic. Again, once they configure these widgets, which again, they're just records here in the database. You can just present them in whichever way you want. There's nothing fancy about this. Again, these are all native FileMaker tools and objects. We've got some slide panels and some stuff in here, but still native FileMaker charts and objects over here just used in a particular way so that it makes it very easy and very flexible. And the last part here I'm showing here is that because I'm using SQL, which is another advantage of SQL, is that it is not dependent on my relationship graph.

And so I do not have any of the constraints of context that is brought to by the relationship graph, which does happen with DAPI, for example. And so this flexibility allows the user to ask for anything that they want because they might see connections later on that we didn't foresee or that maybe we didn't represent it in the most efficient way in the graph. And so, you know, I brought this up here and I showed here what I'm showing here is the relationship graph. You can see that the data tables are actually not even part of it. There's no relationship between them in FileMaker. They're just standalone tables right now. But because it's SQL, right, all those relationships get created on the fly.

This still works, which is really cool. It's a huge advantage. And just a little bit of commentary on how you're not sending the data to LLMs. And the reason I keep bringing that up is because the traditional RAG or retrieval augmented generation is literally about sending your data to the LLMs. And I think the interesting thing about this approach is that you're not. Yeah, there's nothing here, again, no data other than the schema is being sent. Because I think this is one of the concepts that as a developer, I'm always looking, I've been looking at generative AI as a kind of as a logic engine. So here it is really all it is doing is helping bridge that gap.

It's here doing a function. It's not manipulating the data. It's not looking at the data. It is something as a programming language, if you will, because when I'm programming it, when I'm asking it, it's in plain English, but it's sending back a result in a fixed way that I want. So it's really there as a logic engine, right? Something that would normally require a human being to write out, it is doing for me. Right, yeah, which would take a lot of effort, right? I mean, but the other thing that occurs to me is this could apply to other than just like built-in FileMaker charts, but I work a lot with Google charts and I don't know if that would work like with the SQL query, but you can just give it a variable data.

You can give it like a, you can have it build HTML with CSS or something and have it build that in your application as well. Then you get all the other chart types. Absolutely. I actually have a version of this similar to this using Chart.js, which I love. It's really easy and does the exact same thing. And by the way, it's not really writing JavaScript, it's not writing HTML, it is just giving a JSON object with the configuration. My JavaScript takes that, interprets it, and creates the chart on the fly. Yeah, that would be local too, right? It's just a logic. Yeah, like if you use Google Charts, you're still sending your data out, but yeah, with the people running it.

So yeah, let's take a look. Can we see some more examples of this? I guess we saw a couple a minute ago. Yeah, so this here is just showing kind of the scripts. I can see the scripts behind the scenes. It's really, really easy. It's really, really small. There's a main script here, generates the SQL and the settings, and then everything gets run locally. So the entire query gets run on your machine. And that's why it's really that fast. It is very, very fast, which is really nice. And just for some clarity, this was the before this week release of scripts. uh we're going to talk in a moment about some of the new tools that you're at your disposal but just to really drive the point home that all of this was possible even before the big announcement this week and i think the real takeaway there is that um people there's nothing to wait there's no reason to be waiting to try to integrate these features into your solution absolutely absolutely uh you can start using those things today and uh this and the other files that i use used during the session are available on community so you go into community.clarus.com look forward for the session be able to download download the slides as well that you use during the session and just take these files apart look inside see how see what makes it tick

modify it learn from it i really really would love people to to look at this learner and iterate on it make it better we had um we've had already some people do that uh we had uh Nicholas, the other day, took the fine example and made it into an add-on. This was fantastic. I just love when people take stuff. Yeah, you're talking about the Claris Community Live livestream, episode number one, where the feature was taking one of those files, like you said, and they did a great discussion on talking about that. By the way, the next one was one that, we'll put the link in the show notes, but it was one that I did with Rosemary on basic prompting principles, which I would encourage people to go check out.

And then, by the way, there is one coming after that too, right? At some point over the next couple of weeks? Yeah, we're looking to have it. We'll have the link out soon. We'll have another live stream. We're looking at July 11th. It'll be just about AI, so we'll have some people over and we'll have a conversation about some of the new features, some of the things that we're going to talk about here today, but more from a developer's perspective, right? We want to hear from the community how they're using it. How are you guys using it? what do you see, what makes it, what's exciting about it for you, it should be great, really, really good. Fantastic, great resource, looking forward to that, and so you're showing us now how you've integrated into a dashboard, and by the way, Matt loves dashboards, he's a huge fan of dashboards, and so we're actually seeing FileMaker charts interactively, and mixed maybe with some other charts, is that fair to say? Yeah, so here, yeah, so here you can see all different again it's the dashboard you can see all the different widgets when you create a new you know widget here just another record you can choose a type uh what i showed early before it was an indicator but here's a chart this is pie chart i'm asking you to create a pie chart that shows the account of opportunities for each deal in the stage and click on the button um and that see the chart appears in the back and that's literally this is real time this is not

yeah no fast forwarding no editing that's how long it really took to generate that chart but once again it's not sending all the data it's not sending any of the data over it's just sending some configuration and all you're getting back is that little json object and then all the the query gets run locally the cool thing about that is that then these charts can always run it's not static data and so because it's a query right these charts will always be alive which is just great and technically different users can get different experiences as far as the visualization of their data as well, potentially based on job roles or the data they have access to.

And it's something that developers don't have to think about deterministically in advance, right? So I think the era of deterministic coding, meaning we have to try to capture all the possible use cases, weeks, potentially years in advance of actual users engaging with our applications might be behind us. Ronnie, philosophically, your thoughts on that? I hope that a lot of that can move away. Again, I see, especially generative AI, as a developer, right? I see it as an amazing tool to be used as a kind of a logic engine, right? Where it can do things that would normally require some human reasoning. And I can shift those things away and build that into the application and take away some of the things that I would normally have to think ahead way too much.

try to predict how the user is going to how to use it just give them the tools to be able to be completely flexible to do that right this is an example of that i have no idea what type of charts they're going to want to create what type of which data what type of relationships they're going to want to create so this allows me to take that away just built in that logic and let them go with it and i think it's extremely powerful it probably applies to the the ask interface too right so if If we look into the FileMaker 2024 release, so there's a, it's not an add-on, it's a, or is it an add-on? The ask, the chat interface that has also been made available to the developer community as part of the release this week.

Yeah, so it's about asking about your data. You know, I've seen so many iterations of things like that. And I think it's just not, it's not uncommon, right, for people nowadays to want to be able to chat with their data like they're they're seeing all these little chatbots out there and so those things can very much will be built into FileMaker so if you think about how this is built right uh a chart could just be a chat interface as well so you know being able to generate retrieve the data generate that and generate an answer um it's just it's fairly easy to um to forward and if you if you don't want to build that from scratch there is a is it is that an add-on or a file or something as part of the 2024 deliverable is what I'm angling towards here that actually allows for the chat interface in your data. We're testing those things. We're trying to bring in some of those things, but 2024 is not officially there. So we'll keep you posted.

Scoop. Okay. Gotcha. Loud and clear. Yeah. So now I think the biggest thing that we see, if we're looking at a lot of the Claris messaging, it really is about semantic search. Can you talk little bit from a product management standpoint how that decision was made and what that means and uh and and frankly you know in the context of uh file maker searching how's that different than maybe what we're used to so i love semantic search and during my uh uh a few moments on stage during the keynote engage i was talking about you know some of the things that we're working on behind we're playing with a lot of ai technology at clarus right you want to see what what makes sense for us to bring in how and semantic searches keeps coming up right um christian i've talked so much about this as well uh but part of the you know what a lot of people are doing is you know is retrieval augmented generation that reduce the the amount of hallucinations in your interactions with large language models and i'd say maybe the preferred way of doing the retrieval is using semantic search and so that search isn't new right but But it definitely has seen a resurgence now with regenerative AI, especially with large language models.

So I thought that, especially when we're talking with some of the early testers, that this came up over and over again. Like this is the feature that really is kind of the building block for a lot of the use cases that they're looking at. Like the chat we just talked about. People may not know that the linchpin of chatting against your data is the semantic search, the retrieval aspect of it. exactly as far as vectors are concerned we should say so uh how do vectors embeddings and semantic search all fit into FileMaker 2024 in that experience and I thought and I think you guys have talked about it before in the um uh here on podcasts that uh semantic search you know one of the things that excites me what recently excites me is you know FileMaker already has some really good search capabilities right we've had um searching FileMaker is really easy it's fairly easy to use right it it's a search by example you don't have to learn you know any query language and anything like that but semantic search really is about finding things by how similar they are to to other things right instead of being specifically by lexical like the exact keywords and things like that so for instance in semantic search if you were to search for improving customer service

you know it might yield results that include enhancing client support or boosting shopper satisfaction right because semantically the meaning are very similar even though that the word yeah or even opposite it's pretty fascinating right so things that are do the opposite as well that's one of the other use cases right defining outliers in in a data set which is really great and so you can imagine that if you take semantic search and you and use it with the filemaker traditional keyword research and combine those so you can make some really powerful search capabilities in there and you know really built some really cool things but like you said there are other things they can use it for like finding outliers in a data set you can use it for recommendation systems and things like that it just really is really great but where really I think it shines is when you when you really combine it with large language models and FileMaker I think just magic happens it's really really amazing there's some really cool things that happen when you take all these things and bring it together and so at Engage I was actually showing one example during the keynote of using a pre-release version of FileMaker that has semantic search and can do some really cool stuff. And I'm really happy to say that that version of that application that worked only in a pre-release version in our current release works beautifully. Nice. I will try to make this file

available to everybody, but I don't know if you guys want to see it. Yeah. Awesome. All right. So this So it's fairly simple. It's fairly straightforward. What it does, it really is, it allows you to ask questions about your documents. So, and when you say documents, just to get it into FileMaker speak here, we're talking about documents stored in a container, yes? Yes, documents stored in a container. Have they been transcribed into their text representation? Or, I mean, there's gotta be some sort of overhead, right? Or just storage. These are just regular files. So I'm gonna take a regular file over here. I'm gonna take, just to show you how this works for the end user.

I'm going to add here. Nothing up your sleeve. Nothing up my sleeve. I already have a list of documents already in here. I'm going to click here to add a brand new document. And is this to require a special type of document? Because, you know, obviously we have a lot of flexibility from a binary file storage. Any constraints there? Right now, the way I've built this right now is limited to text files, but it can absolutely be expanded to other types of files. It's just really about how to get the data out of it. Since FileMaker has really easy built-in functions to take text files and just get access to the text this was just easier for me to start building those things but um no reason why you couldn't say add pds or something like that so i'm going to just drag this file over here to this container field and it kind of just kind of sucks it in right now i got 24 documents instead of 23. and what happened behind the scenes is that that document got split up into the text was extracted from the file it got split up into a lot of little chunks. And for each one of those chunks, a vector embedding was created. And those vector embeddings are just numerical representations of the meaning of that text. And then what that allows me to do now is that when I ask it a question, let's say... And by the way, can I point to the chunk part, I think is actually a really important part here too. The reason that

you're not just saying put the whole document as a vector, which you can do, by the way, And there are some use cases possibly where vectorizing an entire document as one series of vectors may be meaningful. Like, you know, like let's say you're doing resumes versus job descriptions or something. Maybe that, you know, is sort of an interesting way that you compare needs and solutions. But the reason that we have chunks is to reduce the noise, right? So you may have a 50-page document and the answer to someone's chat question may only be found in a single paragraph on page three. So we don't want to send them the whole document because the information in the whole document might not be relevant to the question.

So instead, what's your... I know sort of the industry standard is 500 token chunks just for lack of a better reason. What was your approach on this one? I actually, because FileMaker has built in, instead of characters, I went with words. FileMaker, one of the great things about FileMaker, we've got 30 years of built-in functions that we've honed over years and years and years. We've got some amazing text parsing functions built into FileMaker. So I actually did it by words, which is really cool. So that way it wouldn't cut off a word or anything like that. You can do it by sentences, by the way, because FileMaker has that built into. Oh, wait, talk about that.

What FileMaker native tool would you use to isolate a sentence? That's fascinating, actually. Values. Counting the periods? Values. Oh, if they're... If they're return separated? Yeah, that's a very good. So you can do that, or there's several ways of doing this, right? I'm sure there's somebody right now listening to this and writing it in the comments right now. Oh, yeah, you would do this. Of other ways of doing it. I actually did it by words. It's really fairly straightforward. So then you do the words, or you could do the sentences. You have these chunks. Now, the vectorization process, I don't want to just skip right over that either. That requires actually sending that to an embeddings model.

how is that let's bring that forward into 2024 how is that happening happening more natively now than like let's say for example back in february the way you were doing it in those examples yeah so before filemaker 2024 if you want to get a vector embedding right that numerical representation you would have to talk to some some external software most likely a commercial application through api so you'd have to call an api send your data out get the vector back over there um start over there and then use that storage over there to do the search whenever you needed it and just to clarify there there are specific models like for example if we just look at open ai they've got the gpt3 models and four and everything but they also have a separate set of now they have three embeddings models for example the one that everyone used to use before was ada there's large and i think medium or small or something like that now and there but there's a ton of open source ones as well too how is how would how are embeddings happening for users of FileMaker 2024 so you got two options you can you can actually use the and we'll go over we can go the scripts and functions in a second but you can call directly open AIs we'll start supporting

other commercial APIs out there but we've got a script step inside of FileMaker that allows you to create embeddings and create embeddings one at a time or you can create embeddings for an entire set of records for you. And you can either do it through one of the commercial vendors out there like OpenAI or if you want to run your own and really enhance privacy and protection about your data and want to keep everything local, you can run your own models, you can run your own kind of local server and get one of the open source sentence transformers embedding models out there and run them completely local. And FileMaker will work with that local server that you have and use in one of those open source models.

So we'll run with either one, which is really fantastic. So the advantage there, yeah, just to give some context there. So you're saying, okay, so first of all, there's something to unpack. If the FileMaker 2024 has official support for open AIs, not only the language model, the general language models that we're familiar with, like GPT-4.0, for example, but also the uh three open ai embeddings models as well too ada and then large and small but not but not and now this is i don't consider this to be deficiency but not necessarily like you know anthropic or you know some of the other models yet i'm sure those are coming just for clarity yeah yes and then i think this is a really important part that people don't maybe not understand is that in order to create the vectors it's not just like a function you have to send it to the language model and that's actually why that's so powerful is because it's the language model figuring out all these different dimensions and uh embeddings models have different dimensions meaning i think the best way to plain english that is that um you could ask for um you're asking for meaning you're saying hey here's a word or a sentence or a paragraph or something and i want to extract features of it or things that describe it or attributes of it or concepts that are that are described in that text even better i like concepts and and the more uh i mean to some

degree the more uh of those you have available more features you know that you have available uh potentially the more meaning you can extract from your data in these vectors exactly um what support like how many dimensions are we talking about like you know you can have up to 4 000 dimensions or 300 or something and i thought i saw somewhere in here there was uh support for a pretty significant amount of dimensions, I guess, is what I'm getting at. It's a pretty good support natively. Yeah, we'll support OpenAI as far as I believe it's 1,536, I think it is, somewhere around there. And just to let you know, our team here, my guys swear that 300 is plenty. I mean, I'm sure that... I was going to say the exact same thing. There are a lot of use cases where, you know, 2,300 is plenty. You don't need that many for every single application. And as a thought experiment, just to kind of for people to wrap their heads around it like uh matt and i did a conversation in a previous uh podcast where i said hey matt describe the word soccer tell me as many features as you can about the word soccer and matt's a very smart guy and he knows the concept of soccer and we were able to come up with maybe eight or ten so to do 300 like that requires you know a scalable processing like this dude 4 000 is just a thing and frankly that there is a lot of overhead the

The more dimensions is the more overhead. It's literally the more vectors, there's more storage, there's more file size. So finding the sweet spot at maybe 300 is actually very beneficial from a decision-making standpoint for a developer. So what all this means, by the way, at the end of the day, it means once those things are there, then we can search by those meaning. We can search what things are more relevant to something that we're looking for. So let's say in this case, if I'm asking for what are the new AI functions in FileMaker of 2024. And I've got here a found set of all of these different documents that include, you know, Claris Connect terms of service and things like that, but also the release notes. If I click here on the submit button, what it'll do is it'll take this sentence over here, this question, and it'll create a vector embedding of that. And then it'll compare it. It'll search for all the embeddings in here of all of these documents, find the most relevant ones, and then send those along with the question to a large language model, which then can generate this answer. And not only can generate the answer it tells me all the new function the new ai related functions but it'll also tell me what the source is it'll tell me which part of the document the ones that i'm looking here it got the information in order to provide this answer so when you're chunking so we talked about you're using word chunks by the way it can you actually have this for those

listening if you actually look uh on you know really closely for those that are watching you can see that he mentions the chunk so what you're saying is uh we searched we took a bunch of documents, broke them into chunks, vectorized each individual chunk, and that is the same concept as sending them to an embeddings model and getting vectors in return. Storing the vectors, we want to talk a little bit about that, too, because that's a big part of the new release. Absolutely. How and where are those vectors stored before we're even doing these searches, for example? So, I'll show you, because it's really cool. So, in... Nice. So, that's not the right one.

So, they get stored, they got a separate table over here just for the embeddings. And so, all the information is stored here. So, we've got here, I've got a field here for the document title. We've got here the number of the chunk. But here is the text. That's all the text for that particular chunk of this particular document. And we're not searching the text just to help people understand this. I'm storing here. We'll retrieve later and then we'll send over to the LLM to answer the question. But the vector itself, the embedding is stored in a container field. Wait, hold on a second. Wait, wait. I know. This is awesome. We're all going to talk at once here.

The engineers were fantastic. They were able to, the functions here are able to store vectors in a text field, which that's completely normal, right? Because a vector is just an array. It's a bunch of numbers. So you can store it in a text field, and it works perfectly fine in a text field. And how is that an advantage over storing it in a text field, for example? An advantage over just having it stored. Like you could store vectors in a text field, for example. Why is it better to do that in a container? So in the container field allows us to store in a binary format. And so it's smaller. And when we're doing the searches, it's actually much more efficient as well.

So it's fast. You get faster results when you're doing the searches. You get smaller footprint as far as the storage goes. But it all works inside of a container field. So we're storing it in a binary format, a little file inside a container field. The great thing is that you don't have to make a lot of these decisions. You can, it'll choose, it'll use either one. So if you point it when you're doing the, when you're setting up your script step to a container field, it'll assume that these are binary files here. If you point it to a text field, it'll assume that they are in text format, which is really cool. That's actually massive from an overhead standpoint too, because let's imagine that we took these vectors, By the way, we said 300 dimensions, right, of vectors.

That's a ton of text. That's a tremendous amount of text. If you actually store that into a field and then want to search that field, you also have to index that field. So immediately your applications would have become exponentially larger just based on storage and indexing. Yeah, you can also use external containers, right? So your database gets even smaller. Oh, so wait, so these containers that store the vectors can also have external storage? You could. Probably not. You'll probably lose a little bit of speed there, right? Because you still have to feel a little bit more reading outside, but it does work with install storage. Interesting. Okay, cool.

And that would like, if you have a large database with a million things, and this is a significant size of data for every single record, then that would be a big difference. But it's good to know that actually it would be faster if you did them internally. Yeah, that's interesting. a little bit. So it feels sort of like an index. Basically, when you create a record, you run this process to create vectors of the representation of that data in that record. Then -- and that doesn't require any -- well, it may or may not require getting that data out to some service, like OpenAI. You said you can run that locally on your computer as well. Then you do the search.

So let's talk about the search part, unless I'm jumping the gun. Wait, hold on. What is not having to get sent out to OpenAI and done locally? Just to clarify, because you can't actually... So you're saying if you're using the local embeddings model option is what you're talking about there? Just to be clear, because FileMaker isn't generating embeddings itself. I just want to make sure that we're not confusing anyone. So it's either using open AIs where you are sending out the language model, like we said, you support the open AIs, or the local, in the case that Matt, you were just describing is you're talking about like an open source model that's running on your machine, for example, right?

Right. Yeah. Like on the FileMaker server or some other computer in your control. Makes sense. Okay. Okay. I just wanted to clarify that. Yep. So once it's there, then we can perform the search and I'll show you the script. It's really fairly simple. This is not, it really isn't. And these are the 2024 script steps that we're looking at here? Okay, so for those that are listening, we'll navigate you through here. But so in general, the new script steps, just to give people a high level, we have a configure AI account, insert embedding, insert embedding in found set. And then the last two are perform semantic find and set AI call logging. So I just wanted to let people know what has been released.

Those are the, just the AI script steps. There might be some other stuff too, but talk a little bit about how you use those. And to make it easier, we create a brand new section here in the script step on the pane here on the right side. And so that's just about artificial intelligence. So all the new script steps are here, configure AI account. Oh, that's an old one. Yeah, exactly. The configure machine learning model, which is about Coromel. We've put that classified in there. We thought it might be good to move it out of miscellaneous and move it into artificial intelligence. miscellaneous oh so sad we got insert insert embedding uh insert embedding and found set perform semantic find and then set uh ai call logging okay so yeah so for creating the uh start creating all these things you first have to configure an ai account and that means providing uh an api key whether that's going to be for open ai or even if it's your own local model you want to still want some security set up there and you can configure that and so this strip step is all about configuring that. Where's your embedding model being hosted? What's the API key? I'll be able to authenticate that. And then you've got a name. By the way, for those that are familiar with

the Configure Machine Learning Model script step, it's really doing the same thing. The Configure Machine Learning Model would take a stored CoreML model, .ml model file, for example, and make it available in that user session. So this is the same thing that it's doing So if I'm one of the many users of this AI embedded FileMaker application, I still, in my session, need to configure the account, right? I'm not configuring the account for all the other users. This is the same thing as we do with CoreML. Okay, good. Exactly the same thing, yeah. And then you can choose who the provider is. Like I said, we've got OpenAI from the get-go, but you can also choose custom.

You're going to have a local where you can run your own server. Then you can choose this custom over here and then you can provide where the endpoint is. Where does that live? Would you suggest doing this at the beginning of the site? So just philosophically, this is the same kind of thing with Cormel. Would you suggest doing this on login of the system or when the user is about to engage with AI functionality within their session? What's your approach? I like to do it at the very beginning. I like to keep all the things that are, that are I'll have a configuration and use them on later, later on just to have them in one place. So for me, the logical place to put it is on first window open.

I normally have a script like most of us that'll trigger when you open the application. And I'll usually have a section there or I'll call the script that does all these configurations. So there's no overhead for it running in the background that we would need to be concerned about. And then secondly, if I try to do one of the supported script steps like perform semantic find before this was done, do we get an error message? Yeah, you'll get an error. Okay. And there's very little overhead of having this one because all it's doing is one API call and then storing that response. So like a handshake, basically. It's a handshake, that's all, yeah. And by the way, that's kind of cool because that's the first time we've actually had any kind of a script step in FileMaker that directly does an API call without any extra hoopla, yeah?

You hit on it, that's absolutely right. Yeah, good point. This is one of the things where we're making a lot of some different things here. It's calling for a lot of different, really great approaches here. And that's meaningful in the old, like the ones you used to be doing prior to 2024 were all insert from URL, like API calls to language models. And so now you've kind of like broken up the insert from URL into various different steps to make it easier for users to be able to do that. And this is a great example of one of those steps that you've done. And for what it's worth, you can still use insert from URL script step if you want to talk directly to the API.

Fantastic. We're not taking that away. We believe that there's a way to make this a lot easier for users, right? It's all about reducing the amount of effort that it takes for you to bring these features, power features by AI into your applications. And so we're just bridging that gap and making it a lot easier. Can we take a look at that perform semantic find step? Absolutely. So I'll go, so this script here is the one that really answers the question. And so here I'll walk you through, I won't go through every single step, uh you know some cleanup here at the beginning get some preparation uh here just setting things up to find the chunks that are relevant to the question so this is the the native file maker part of the question yeah retrieval part okay all it does so i'm doing here i'm go to uh related records i'm just kind of showing some of the during the records there but here's where the performance semantic fine happens so i'm going to search here for all the um for all the embeddings right across across my table that are semantically similar to the question that was answered and here's an interesting gear icon on that sorry ronnie just uh and and and i i just i think this is really important for those that are new to this but you're calling out an embedding model

in the perform semantic find it's you are actually embedding the question at this point is that correct and and not the answers that have already been embedded so at the point that a user is asking a question all the documents have already gone through the embedding process you're now embedding the question which i think is kind of a hidden thing that people aren't really that people don't know is happening in chats talk a little bit about that yeah so we're doing two things this script can do two things right it'll take uh and it'll depend on here on what type of query is and we'll go this one which is kind of the easiest one what we're doing here which is query by natural language what this means is you give it some text it'll it'll create the vector embedding for that text and then use that embedding to find all the other embeddings that are similar to that one, right? Because none of that's the way it happens. And so, you know, give it what the count that we want to use, which model we want to use. And then here's the text.

This is the question that was asked. This is the text that we're going to create a vector embedding for. And then where are we going to search? We're going to search in the found set. And this is the vector embedding container field that we're going to search in. And we made this on purpose. You can either search through all your records or you could search through the found set and the reason is you may not want to search through all of your records it's possible that you may do a semantic find after you do a traditional filemaker find you might reduce the amount of records by searching through categories maybe we're searching only for support documents or maybe these are only legal documents maybe these are hr things and because you can reduce the amount of those records and then perform the find only in that current found set.

And the advantage is speed is one of the big ones. And for those that are listening, you set this up with the go to related records step that precedes this. So you can leverage existing relationships and queries to be able to establish those contexts within your scripts. And so, yeah, there's actually, that's a huge point. Like a lot of intelligence is built into the found set. So you're really kind of constraining um and and i in the other part too for those that might have a little bit more experience with the ai stuff this is actually uh like a re-rank so uh if you look at you've created you've created a found set already in filemaker using a lexical search which is index based or relationship based like you've done here with go to related record then within that found set you can actually do something as well too so you can reorder which would be a true re-rank but now you're actually searching within, which is kind of a fascinating approach. Yeah, that's cool.

There's a couple of other things I love about this, too, that I'm not sure if you're going to remember pointing out or not. But when you get the results back, they're ranked. They're sorted by... Yes, they are sorted. So the ranking is key. The ranking is there. That's really hard to do. That is the re-ranked part. Yeah, specifically, that's huge. Yeah. So if you ever use any of the other vector database that has metadata, a lot of them provide a way for you to include in the query some sort of filter information so you can you know query again by category or something like that and so that will again that reduces the the the set of vector embeddings that you're searching for this basically equates to that so if you're doing a search to found set after performing a search a regular file maker search right a traditional file major search it is basically the same thing right you are going to reduce that and you can make it a lot more a lot more relevant because you might find that maybe you're looking for support documents per se, right? And that what you're looking for, if you didn't do this, you might come up with something from HR, but it might not be relevant to your particular use case, right? Maybe they are semantically similar, but it might not be relevant for you. That's managed by return count here, right? And so those that are doing like this outside of FileMaker, this is also what's

Determined is the threshold of your search. So sometimes you can say, give me the top, the most similar results from my vector search. Talk about how that's similar and a little bit different with return count here. I know that's kind of a big deal in this process. So return count allows you to specify how many records do you want back, right? Because you might... So not the threshold, not the similarity threshold, just to be clear. Not yet, not yet. The return count is how many records do you want back? And you'll get your typical calculation. But over here, and I'm not using it here in this particular case, but here then you do have a threshold. And so I can say, I want only those.

For those listening, it's the cosine similarity condition option that's in the perform semantic search script set. That's right. Yeah, yeah, yeah. We added several logical options here. Greater than, less than, equal to, greater than or equal to, less than or equal to. And this allows you then to specify in the cosine similarity value, What value do you want to compare with it? So I can say, only show me those that are greater than 0.7. And now pause for a moment because cosine similarity is a new concept for people too. This is a great time to bring it up. There is a function called cosine similarity, which literally just takes the cosine. Well, you can talk about what that means based on your question, let's say in this case.

And then the cosine of the answer that comes back and tells how semantically similar those things are. And there's a function to do that. How does cosine similarity work in the search function context that we're looking at here? So there are several ways of calculating how similar two vectors are. And cosine is one of the most popular ones. We decided to go with cosine similarity. So built into FileMaker, there are now functions and there's capabilities of calculating cosine similarity between two vectors. And so what this script step is doing here is using those new capabilities in order to perform that find, to reduce the amount of records that it finds and finds the most relevant ones.

So I can say, hey, give me those, only those that have a cosine similarity that is greater than 0.7. The cosine similarity returns values between minus one, where something is very, very, very different, peg um to one where two things would be extremely alike or almost identical zero means that they have absolutely nothing in common you can go either way these are uh represented as uh zero point whatever values um often you may see these as discussed this is not yeah like uh yeah i mean you probably want to round those but a lot of a lot of times people talk about these is percent similarity and it's not wrong but you know i think it's probably a good way to you know sort of plain english it uh but it's very similar to like in uh core ml for example when when you're doing like you're sending an image over to a trained model in core ml you're actually getting back every single possible classification of that image with a also a zero point value in those cases you're getting an array where the entire list of classifications they all have a value that and then together they add up to 100 that's not the same thing here right

uh just to kind of to compare and contrast yeah this is again this is just saying what the the value of the calculation between these two things again looking at the different concepts that are captured in the embeddings how similar are the r vector one and vector two and in this case the key is the closer they are the higher that number or closer to one it is let's put it that way right the closer to one man talked about re-ranking this is the thing that you were talking about like when you're sort when you're sorted by this it's by that cosine similarity yeah and so the results i get back here in this case i'm asking for i'm asking for 10 those 10 records will be sorted by this cosine similarity where the top one the first record in that found set will have the highest cosine similarity value.

So, you know... Okay, there's a question I have to ask. Go ahead. So if you have a found set of 100, or you have a found set of 10, like if your return count is 10 or 100, are you going to still get the exact same 10 records at the top, or could it be different? Explain the question, sorry. So if you configure this two different ways for the exact same search, one of them is the return count of 10, and one of them is their return count of 100, you're going to be looking at the first 10 records. Are they going to be the same, or could they be different? If you're searching through the entire database, you'll probably get the same ones at the top. Yeah, that's OK.

Yeah, so that's kind of the way to tune this. So what about the performance of this? So if you're searching on 100,000 records, is it going to have to actually read the container data? I guess one of the other things I really want to point out here is this is all happening locally. This, to me, is the magic part, right? But if you wanted to do this before, you had to actually send a bunch of data out, have the query on some other thing, or have your data, you know, sync to some other server in a non-filemaker database. Well, just to be clear, it had to go out at some point for the embeddings to be created. Right, but now it's local, right? So you can, you know, once it's… Well, but the question needs to be embedded as well, too.

So if you're choosing OpenAI, that does need to go out. The local part, there's two things, I think, when you're saying local. You're saying the vectors are stored as binary in the containers. That is truly local. But also, you could use an open source model that a lot of people refer to as local to do this. So when you ask the question, you don't, like, for example, the application doesn't know what the user's question is. That still needs to be embedded. I get that. I get that. But what I'm saying is like once the question is asked and once the embeddings are done and it's comparing the question embedding to the data embeddings. Yes, just to clarify, yeah.

Yeah, so is it then like looping through all of those records? So how would you, what's actually happening and how do you optimize that? Those are my two questions. Okay, I think it would be good to bring one of the engineers to go through all the underlying technical aspects. But it is going through each one of those embeddings. They've got some really amazing magic. They're loading a lot of other stuff into memory. I think the cool part, though, is if you're running this hosted on FileMaker Server 2024, we actually leverage FileMaker Server to do a lot of that work. They do some really amazing magic where depending on how many records you're looking at, how many records are already downloaded to the client side and things like that.

Part of that find will be offloaded to the server to perform. And then you only get the results, which I think is fantastic. So there's some really great benefits of just having FileMaker server and have a good server and it'll have the hardware to support all this, all this stuff. But yes, it will happen here locally. It notes most of it in memory, but it'll try to leverage FileMaker server whenever it can. Ronnie, is that just on cloud, FileMaker cloud? - No, FileMaker Server. - Awesome. - FileMaker Server, yeah. - So a great reason to go upgrade your server to FileMaker 2024 and really-- - Matt, I wanted to show you this, Chris, 'cause I think you'll get a blast out of this.

One of the options over here, like what I was using here in this particular application, because there's text, I'm just writing the text there, and you don't know, like you said, what the text is gonna be, the option here for what the query is is natural language. That means give me text, We'll convert it into a vector embedding, and then we'll find things that are similar to that. However, there are cases where you already have the embedding. So for example, if we have a document, all of these documents have embeddings. What if we have one of those documents and we want to find other documents that are similar to that document? There's no reason to re-encode that, to re-embed that document.

So we can actually choose here vector data, and then we can choose a container field that we're going to use as a search criteria, and then point it to other container fields where we're going to search for similar ones in there. Because I know, Chris, you and I have been talking about some of the use cases here where that would be involved, right? So recommendation systems, finding, pairing up. this type of this option over here enables you to do that fantastic yeah that's a huge that's a huge deal and then in either case you're getting a return count and um and so this is similar but different than confidence for example uh just assuming maybe people have worked with coramil before you're getting back a confidence of all the potential classifications here though it's a found set and technically if you say return count 10 unlike with confidence where they will be the percentage of uh how confident the model is that it's identified or classified something here technically you could have a situation where you really have a thousand records that come back as potential matches you're you're constraining it to 10 they could all technically have like a

0.89 uh similarity they don't cascade like a confidence does i think that's probably an important thing to clarify and that yeah and that's where that's where adding the cosine similarity condition comes into play if you want to make sure that these things are highly related or more related or have at least a minimum threshold of similarity then you would use a cosine similarity condition and say hey i don't want you to bring just which ones are the most relevant regardless of how relevant they are i want yeah like a really really poor search can still yield some records it's going to find something like you'll always find something but it might be restless now bear in mind if you if you add this if you add this condition there like you said maybe there aren't any documents like that and so it's possible that this condition of 10 won't be met maybe there are no great maybe there's only five right condition and so it was really both of them together and this is and then so like for example in the core ml support you have the return at least one or the top one so all you're really caring about is the top one i will just say from out in the field uh building and deploying these applications a lot of our bots only pick the top one like our threshold is give us the most relevant and there is a very much a science to how much more you want to return so this is not arbitrary this should be taken very

seriously and this comes from testing you have to go do searches you have to have your test and your ground truth and that's how you figure out that threshold that is not a random just pick whatever and go ahead this could be that absolutely is a a lever that will help you get good quality results from your semantic searches or not so just to kind of point to how important that is yep what's what's the what's the speed going to be if you're searching against say 100 records or a million records um it depends uh it depends on on several things whether you've got them stored uh in text If you've got them in embeddings, if you're using server, what you have on your machine, right?

Hardware here is very important because there's a lot of calculations that happen in there. So as an example, if you've got an Intel-based machine on a Mac versus an M3, you can have different performance for sure. But I mean, are we talking like if you're searching on a million or 10 million records, like a minute? Or is it just going to be seconds or like, you know, in general? I've never tried with a million, so I don't know. But hundreds of thousands could take several seconds. Okay. But still, that's still pretty fast, right? Oh, yeah. And if you're on a normal hosting environment with a really good cloud server, like an AWS server that can boost its processing as it needs to when the demand hits.

That being said, I've seen in some of our testing, we had one of our engineers do a search. He was here in the US, we had a remote server. He performed a search on 43,000 records and that happened within a second. But again, it all depends on what the hardware is, what your network, there's a lot of conditions that come into play for sure. But the more records that you're searching all at once, obviously that's going to increase how long it takes. So Ronnie, when you're doing these types of calls through insert from URL, we are all very familiar with the message that pops up that shows the progress of that API call and the return. Is that the same or is that different or has that gone away or what does that look like in the new paradigm in FileMaker 2024? That goes away. That's one of the great things about creating scripts that are specifically designed to talk to APIs and we can do things that we couldn't do or are not there yet in FileMaker. And this is a great example. So this script over here this is the one that calculates the embeddings for the document and so um i'll walk through it's very short you can see there's not that many yeah you know there's more white space here than there are lines of code one important line right there but there's only one important line here which is insert embedding and found set it does exactly what the name implies it's going to create vector

embeddings for every record that's in the found set so again i give it the the name of the account that i'm using so it knows where to call and what the api what the embedding model is and all that fun stuff then i'm taking what the source is hey you're going to be taking this text field and then you can create embeddings and then you can put it into this other field or this container field now as you can imagine this is making behind the scenes making api calls making a lot of api calls right because i could have i can have a thousand twenty thousand records over here but it's doing so in a way where you're not going to see that that dialogue chris that you're talking about We're doing this almost asynchronously behind the scenes, which is fantastic.

You can almost leave this running. It's really great. Are these server-side supported ScriptShops as well? Yes, they are. Oh, so that's the move, right? I was getting there, yeah. You can have this running on the server there as well. So if you, especially for creating the embeddings, maybe you've got, if you have, maybe you already have embeddings from a different system, maybe you're using something else, they're already in text, but you want to convert them, we have functions in this release that allow you to convert from text to binary and binary to text again. So if you already have those from something else, you can convert them into the binary. And again, you can offload those to the server and have those.

As long as those are supported for the code. So when you're talking about cosine similarity, I think we can refer to that as like, that's a specific type of function, let's say, that's used to be able to determine similarity. If you have run embeddings that do not use cosine similarity, You wouldn't want to port those over, though, I think just as a cautionary advice. You would have to run embeddings that support, let's say, call it support cosine similarity, I think is the way to put that. Yeah, I think the biggest requirement that the engineering team has put it here is that you want to use embeddings that have been, I forgot the word right now. so there's process for embeddings where you uh where you take away um uh the length of the of the vector itself um you normalize it that's the word i'm looking for so you want to look you want to you want to use a model that create a normalized embeddings most of the ones you're going to be out there commercially available opening eyes one of them they are normalized i don't want to get down to what the details of what that what that means uh that for the vector it means that you're just reducing the length of the vector to one just to make the math easier so but if the point here is that you if you have already run embeddings and you're using euclidean distance for example you

cannot port over those vectors into your file maker application you will need to re-embed those with something that supports cosine similarity as an example of yes it and if you're just doing it from scratch cool this you've already got all that built into file maker it's going to help you do that yeah the other thing is that it's you can't really mix and match um or you shouldn't mix and match models right yes so if you embedded all these with say you know uh you know text embedding eta zero two when you're doing the search you need to use the exact same model right to to embed the query that you're making there if you use a different model you'll get some really in the future like a different more powerful model could come out you would you'd basically just replace all your embeddings with these models, which are in the search?

Yeah, okay. Okay, okay, I have another question that's burning. We're talking about embeddings. Just to be clear, you can mix and match whatever models you want to speak to if you're not using embeddings, not to confuse anybody on that. So, yeah. Okay, okay. You're talking about the embeddings. But I think what's important here is that, yeah, you're creating all the embeddings, like all the embeddings here for this. This is all the code I had to write. If I had to do this with insert from URL script step, I mean, you know, this script would have been very much longer. Just handling the errors, right? Just handling the errors alone, right, would have been this amount of text, this amount of lines of code.

But, yeah, we take care of all that for you. That's cool. I mean, this is kind of the magic of what FileMaker has done since the beginning, right, is taken things like this one and made them kind of magic, right, from the export or create PDF and automatically send the mail to a lot of other things like that okay my burning question um go ahead so the example that you just showed you're searching against like a container field that you imported like a pdf or text file what if you have a record that has like uh an account with a bunch of people who work there and a bunch of invoices and line items for the invoices and you want to find like i want to find the most similar account or the most or something in that particular data Do you, would you wrap that up into like a JSON object and then embed that whole thing?

Or what's the process for searching? Or how can semantic search really empower that for the user? So I'll give you, let me, I'll actually give you an example. How about that? Yes, yes. If you like, give me one second and I'll pull it up. Because one of the, there was actually a question that came up and one of our test scripts, ETS, on that, right? How do we do certain things? What's the best way of looking for some of these things? Give me one second. So we can do a frame it, give you a chance to look that up, frame it as non-document searches, right? So what we've been talking about and focusing on here is taking documents, turning them into vector searchable data, using the embedding process.

Now we're talking about raw text, And how can we make that searchable in a semantic manner as well? Yeah, text, numbers, dates, the typical type of data that you'd have in a FileMaker table. Or multiple tables, right? So relational is the other thing. Yeah. All right. So here's an example. I've got this table that has just a bunch of sneakers, right? So just imagine these are retail items and things like that. And I've got semantic search. And I've got semantic search that's actually performing two semantic search in two different ways, or semantic search for two different pieces of information. They're the exact same ones. So in here, I'm searching for running shoes.

If I just put running shoes, it searches for both of them. And just for those listening, what we're seeing on the left-hand side, you're actually seeing embedding using plain field values and then embedding using descriptive values. You want to frame up the difference between those two things? And then the result, unexpectedly, is a bunch of cars and pigeons and things that are not running shoes. they're all running shoes but i'm not going to search here um so what's what's happening here is on the left hand side it's just using the plain fields it's looking at the values of the field with nothing else with no context price description name that kind of thing yes with no context whatsoever on the right side it's using a more descriptive field that's taking all the information from the rest of the of the record and providing some context around it so that's exactly what matt's talking about in his example so i'm now i'm searching here for black running shoes. That's pretty good. The search results are fairly similar. But now I'm going to search here for low-rated sneakers, right? And mind you that most of these records don't say sneakers. They say shoes, right? They don't say sneakers. And so now- Or trainers. You could say trainers and it would still know.

Yeah. Or something like that. And what happens here is that the results are fairly different. On the right side, you've got things that are very high, like 4.7 and 4.6 and a scale of one to five right and the reason is the data that i'm that it's searching for what was embedded here is these fields that you see here just this data i just took the those values just as it is right there and just created the embedding for those so for those who are listening it just says you know just the name of it has the name of the of the of the item says beige shoes 60 in stock 60 again 3.9 if i were to tell you you're concatenating all those values together in a calculation or something okay okay and ready those and if i were to tell you matt this shoe is 3.9 with nothing else you have no clue there's no context you have no there's no meaning behind it the same thing happens with embeddings like there's no context i can't tell you what is similar right and so these things is really hard to find something there now in this case on the like that sign you're getting better results because you're taking those exact same fields and what it did was create a sentence that wraps it around wraps around all that data and gives us some context so it now reads you know these super saw mules are baked shoes

prized at 60 that are currently in stock they have been reviewed 60 times and not favored by customers right and you used a llm to do that for you right Oh, so you're -- okay. But I think that's important, though. So while you could use an LLM to take those non sequiturs in that list and create a sentence of it, you did this in a fancy FileMaker calculation locally. Correct. Yeah. So the text we're looking at on the right. Was that actually a previously generated query before you did all this that's then stored in the database somehow? Or is that something that's actually happened for all of those records when the search is performed? No, these are pre-calculated values.

So I'm just taking those fields. What you see in blue in the sentence, those are the actual field values. And so I created a calculation field that concatenates some text with those fields and creates this sentence. Got it. And just as an example of another way you can use an LLM, You could have absolutely gone in and said, hey, language model, completely independent of embeddings, just because we've been talking a lot about embeddings. But it absolutely nothing to do with embeddings at all. You could just take those those the bullet points that you were showing on the left, send them over to a language model and ask it to write with whatever prompt instructions you want.

Write the summaries and do that and store it as text. Instead, Ronnie, because he's got superpowers and FileMaker, has created a calculation with that same logic to do the same thing. Okay. That's cool. Yeah. So that's exactly kind of what I was trying to get at is, is the power of, of kind of combining two different things, right? So like a summary feature in a language model with an API call with this native, you know, semantic find feature to get really, really amazing results like this. So would this work on names, addresses, number values? Like what are the things that's going to be good for? Obviously, it's great for things like this. What are the things that's going to be good for and what are the things that's not going to be good for, do you think?

I haven't found things that are not good for. Again, as long as there's some context around it. And if you give it something like 60, 3.9, there's no context. There's no meaning. Right, right, right. but if i say you know this is a highly rated item uh that's been used many times and that that has a different meaning and so in the in the multi-dimensional spaces of concepts right this can be a lot easier to search yeah in this case though almost almost anything can be yeah almost anything could be converted right into well yeah but we're looking at english words with things Right. So what if it's a bunch of names like first names and last names and addresses?

And we want to find like the closest match that might include, you know, misspellings or things like that. That's going to be to me. I would think that would be a lot harder to do, but I don't know. Yeah, I don't know how that would work, because, again, we're semantics really is looking for meaning. Don't know how that would apply there. Yeah. So I mean, I guess my question is, maybe we'll break it down by syllable or similarity and how it would sound. and maybe it would actually build some of those meanings in there. Anyway, I'm going to try it. Interesting. Interesting is the case. I'd love to try it. Let me know. I would just add in the context of what we're talking about here, just to kind of point to that again.

One superpower of language models is to do these types of summaries and extract this type of meaning. So just because we haven't really given many examples of just how you can use the language models themselves be way beyond question answering. You can do exactly the type of thing that we're talking about. Take a bunch of data, give it some instructions and tell it how to create descriptions. And sometimes those descriptions might not be for users to consume, but they could be something that you're using for embeddings to be able to increase the performance on your semantic search as well, too. So just something for people to think about, because that's actually very common in the non-filemaker AI space to do that type of thing.

So now, oh my God, right? So, okay, we've looked at configure AI account, which is a big deal. And I think it's an even bigger deal that we're not seeing in any of the demonstrations for those that haven't seen it. We have not seen a progress bar of any kind of any dialogue that's popped up. That's a huge, huge deal. You also showed us insert embedding, insert embedding and found set. We've talked a lot about how meaningful that is from a found set standpoint. And all of these have been executed using perform semantic find. I did notice that you were following your perform semantic finds with set AI call logging. Is that a good discipline or what's actually happening there or what wouldn't happen if you weren't doing the log again?

Yeah. So one of the new scripts that we've got is set AI call logging. And, you know, when we were testing these out and the feedback that we were getting was, well, because these are making API calls, which is new, right? It's new for us to have a script step that does behind the scenes, does API calls. That's huge. Having, yeah, having some visibility on what's happening behind the scenes is really important because we're hiding a lot of the complexity. Well, when you're troubleshooting stuff, you really need to see what's going on and see what the results are, what you're getting back. What's being sent is almost as important as sometimes even more important than what you're getting back.

Well, from a troubleshooting standpoint, it's critical, right? Like, you know, there's nothing so magic here that you can just turn these things on and let her rip. As a matter of fact, you know, reporting from out on the street and how these work in non-file maker AI. We mentioned record counts. That's not arbitrary. Just really want to drive that home. Your threshold that you're establishing of cosine similarity and your returns is extremely scientific. It's different from one search use case to the next. People need to take that seriously. And the way you figure out those thresholds are literally by testing, rigorous testing, doing a bunch of searches and having humans or hopefully subject matter experts or human in the loop, look at it and go, yeah, you know what?

One of the things you can do, for example, let's say you're doing translation semantics or cosine similarity. You could say, here's English to Spanish translations. You could do 100 of them. Send them over to someone like Ronnie, who is bilingual, and say, Ronnie, could you grade all 100 of these English to Spanish translations from a one to five scale, let's say. And then behind the scenes, you could correlate what the human in the loop said were high quality translations with what your cosine similarity threshold is. And you might look and go, wow, all the ones that Ronnie said were four star or higher are all from 0.78 threshold or higher. You would then take the knowledge of that and go build that into your semantic search script step if, in fact, you were using the cosine threshold there as an example of why that is so important and non-trivial because if you just set it as 10 or you just keep it at the same threshold every time it's going to your your quality of your output is going to be dramatically different so i just think it's important for people to know that as developers we're taking on the responsibility for the outputs in those cases so it's creating significantly more work for us rather than taking it away which is a good news it's a well let's just put this way it's even more work if you're not using the file maker tool set to be able to pull that yeah yeah well i guess

you know regardless about it is it's allowing us to leverage of what we've built and do way more for our clients the upside is so worth the extra work there's no doubt about it um and then so ronnie like uh on a function standpoint we talked a lot about cosine similarity that's literally where you're taking it if i if we were to use the uh the translation example we would take the vector of uh for the the english word uh phrase and then the vector for the spanish phrase And it literally just says, give us the distance between the two of them, which is really the different, the literally a mathematical difference of those two. That is what the new cosine similarity function does in FileMaker.

The only other functions, and I'm not belittling them by any means, but just so that we're touching on them, they're all get functions, right? Get embedding, get embedding as file, get embedding as text, get table DDL. So what's going on there? Hello. So let me, all right, let's bring these up. And so just like we did with the script steps, we have a brand new section here called artificial intelligence. Everything that has to do with AI is put in the same here category, including those that we had before, right? Like the compute model, which was there before. But you've got, like you said, cosine similarity takes one vector, takes two vectors, right? Vector one and vector two, which can be text or can be containers.

And it will return a value between minus one and one, depending on how similar, how closely they are from a semantic standpoint, right? Those two things. The other ones, just because they say get, they're not, you know, they're not your standard get, right? These are… These aren't the get open parens. Exactly, yeah. The other gets, actually. The other gets, yeah. Yeah. So we got some really interesting ones in here. So I think one that's going to be interesting just to so everybody here is aware that we've got a function here that can literally calculate the embedding. So we're giving you both a script step and a function that can get the embedding, that can calculate the embedding.

And the reason we did this is that there are situations where you might want to create the embedding immediately, for example, in maybe in the auto enter. Maybe you want to create it in there and make part of the calculation. What? Or you can use this function inside of a set field. Not auto-enter. Come on. You just freaked me out. I'm just – I'm not saying you should. I'm saying you could. It's possible. Yeah, yeah. It's possible. I mean, everything – a creation of a record should be script-based. So what I mean is, obviously, when the script finishes making the record. Naturally. But, for example, inside of a set field. Yeah, perfect. You can use this calculation in there, inside of a set variable.

There are different situations where it makes sense to use a calculation instead of a script step. So we have that available here for you. So you get to choose depending on the situation what you want to use. And then, again, because you might already have embeddings from talking to APIs, you might have them in text fields as text. So we've got two functions here that allow you to go from text to binary and from binary to text, which is really great. so you can convert those back and forth anytime you need. The get model attributes is a core model. It's been there. Yep, you've been there. I'm going to skip this off real quick. Wait, wait, wait, wait, wait.

So get token count is really useful because this allows you to calculate how many tokens something might have, which I think is really great. I wouldn't say it's useful. It's a critical component. You have to have something. You have to be able to calculate this because especially if you're talking with an API, you want to know how much this is going to cost you how big this is there are so many things that you want to make sure that you understand what you're doing and especially inside of a script this is a great way to um this is a great function to have so you can start looking into making sure you got everything the right and by the way on the logging there's a verbose is that also doing the token count because token counts and logging is critical.

Yes, it'll give you all the details of all the calls, all the API. Number of tokens used. Everything. Basically, it's a raw dump of everything in a call, which is really great. Sorry. A big deal, important discipline, and the reason I'm interjecting there is because when you start playing with language models, especially for the first time you may not realize how much tokens you're using until your bill shows up at the end of the month and for those of us that are building on behalf of our clients you have to take the testing and again testing i know shocker the testing is an extremely valuable phase of this for you to be able to establish expectations for your clients for in production token use i can just tell you from out in the wild that we'll do an experiment and then the very first thing the client wants to know is is how much is this going to cost me to run this in production and the only way you can do that is to figure out your token costs and your account there both from a literally dollars cost and then just like you know um processing cost and speed cost and different things like that too they all they're all affected yeah now okay get table ddl um so before that so while all of this here has been really about getting the about really about

semantic search about really talking about doing semantic search and all that fun stuff even if you're doing things with a insert from url script step you still if you want to talk to a to a large language model like say for example you're doing the smart dashboard that i was doing you still have to send the schema of your data right and right now sending that schema especially if you're doing with SQL. It's kind of cumbersome for FileMaker developers. It really isn't a really good way of doing that. There are a couple of ways we could do that, but you still have to create some calculations and functions. So we thought it'd be great. So we think that this is going to be a very common application where you send your schema over to NLM for something like that, that we'd have a function dedicated to doing exactly that. So what this function does is it It literally creates what's called a data definition language, DDL, which basically is a SQL statement that describes a table that you have that you're asking for in there.

And so they make it a lot easier for you to bring that information and be able to send it to large language models. And by the way, it's one of those helper tool, right? They make it a lot easier for you to send those things over to them. Well, this is huge because if you try to send your actual DDL or your FMXML, for example, over, you're going to crash it. Your token limit on those is insane. So you're actually creating a cleaned up or pre-processed version of this. I think it's probably the most important part of that. That's sweet. So yeah, it's one of those things where it's going to be really nice. You can send more than one table name in there. So I'll group those.

So there's some really great applications here for it. I think it's one of those things that people are going to be using a lot. Yeah. So we just thought it would be great to have that in here just starting with this release. That's great. Huge. And one last question on the new stuff, Randy. Just in general, we talked a lot about how you have OpenAI support, probably other models soon to come, maybe as soon as next week. I don't know. I'm not making it. I'm just being silly. But there may be other models that might be closer to the family, let's say, that might be supported. Who knows? Whatever, just speculating. But when you're talking about open source, what specifically through Hugging Face, is that what's happening there?

Not necessarily like something you have installed locally, just for clarity there. I think that's really important. Yeah. And we've got a, we posted a technical engineering blog that goes through the process of installing it and also listing all the models that we've tested internally, the open source embedding models that we've tested and make sure that we know that they work and you'd be a little bit more confident in using yourself. But yeah, these are models that you'll be able to download from Hugging Face and start using them yourselves, which is really great. Excellent. And so the options, the LLM options in version 2101, at least, are OpenAI and specific Hugging face open source models um and there's support both for mac os ubuntu and windows that's also important to know as well since those are locally uh run and i guess there's a python environment i see some dot pi uh files that are run here so we're at so so is that on the file maker server side there's actually a python environment that's running is that so yeah what we're doing we're including some Python scripts to help people do the installation of the required software to run these.

You don't have to put it in a FileMaker server. You can put it wherever it's accessible, but you might want to put it on the same servers or leverage the same hardware that you have in there. But giving you everything that you need to get up and running, installing software that will allow you to host these models there for you. Man, so, okay, wow. So, all right. So what a week and what a conversation. Ronnie, what's next? What are the plugs here? Where can we send people for more information? Where are you going to be talking about this? Where are other folks at Claris going to be talking about this stuff? I think that I'm really excited. We're going to have another live stream on July 11th.

We'll be having a panel of developers. We'll be talking a little bit more about it. We'll give people some time to play around with these things and really talk about how they're currently using, how you guys are using it out in the field. Let us know how you're leveraging semantic search inside of FileMaker, What type of applications you're using? Really, how is it changing? And what is it enabling you to do that you couldn't do before? I really think that that's the thing, right? This is really opening the door to do things that we just couldn't do before as developers, right? As FileMaker developers. Or were fairly difficult to do in FileMaker before. That's huge.

That's a community live, just to be clear. Yeah, community live. And that's a new series. mentioned that there was already two community lives. One was showing a modification of one of your demo files for semantic search. So that one I think is an important one to go check out. Find that one on FileMaker or Claire's community somewhere. I also did one with Rosemary that followed that just a couple of weeks ago, which was about basic prompting principles. And I focused on not only prompting principles and techniques, but also how you use your data in there as well. So those are not only useful when you're interacting with chatbots, but also with the script steps that we're talking about here as well, too, as you're putting your prompts together.

The next one then, actually, I think there's another one that's on a non-AI topic, but the one to follow that is going to be related to what you're talking about. And I'll just, I just want to say thank you to everybody who helped vote me in for the Summerfest Tech Conference. I will be speaking at Summerfest Tech on June 26th in Milwaukee, Wisconsin. I'm really excited about that as a native Milwaukeean. I will be speaking on a updated version of my AI deployed where I'm showing a bunch of cool use cases of what people are actually using AI for out in the wild. Some of them are semantic search, but some of them are just kind of mind-blowing multimodal inputs.

And then finally, all of us were on a conversation. This is just a plug. I don't want to, we don't have to rabbit hole into this, but we were all on a conversation together the last time where we were talking about this Claris AI learning initiative, which we have now dubbed, Matt is giving me the business on this, but as Claristotle. And so when you hear the term, when you hear Claristotle, it's supposed to bring a smile to your face. But also that is the working title in the show notes will be an open invite to our Discord channel where we are going to move the conversation and talk about next steps. So I encourage everybody to go check that out. Matt, where are you and what are your plugs and where can people find you maybe even later today uh so uh yeah if you're in berlin at the conference well it's uh 7 p.m here so it's dinner time for me but um there's one more day left in the thought fmp conference let's see and i've got what other plugs do i have they're kind of far off right the next two conferences in europe are october and november um so who knows what's going to evolve yeah you have plenty of time to redo your search uh tool yes and speak on using semantic search i'm ronnie i'm trying to put mad on the spot well actually i

took your advice to put him on the spot and i submit i am going to be speaking on semantic search yeah at those conferences and i'm also doing training classes so i guess i do have a couple plugs i'm doing a uh training class before each of those conferences the day before so um on well we might change the agenda a little bit but basically on a lot of the best practices advanced json stuff and semantic search integration nice and then i also have my um the full class again starting in early august the uh biomaker champion class uh to to help really take people who are internal developers and can't be in the use of FileMaker but don't have a ton of dev or they have some dev and they want to like get up to speed um yeah I love that class it's very hands-on very tutorial based and it's going to be even more uh even more AI focused in terms of actually the teaching I'm going to be using uh yeah I guess well I'm stopping back myself up a little bit you and I talked about a book recently from from uh salman khan which we got the title wrong it's actually brave new words and he's

talking about his um his use of chat gpt as a teaching assistant really as kind of a private tutor yeah conmigo yeah which is a play on spanish conmigo yep and his name khan and well and also We talked about that in the Claristotle. We were talking about the new era of learning is to have this guide on the side. And never has that been more realistic or possible than now with AI tutoring. And I literally just taught a class, a private FileMaker class this week. And when I got to the point where I said, well, have you guys used ChatGPT before as a guide on the side? Even though we hadn't raised the latent intelligence, which is what the Claristotle initiative is all about, even at whatever accuracy it had, the way that the learner, these new learners reacted to this, like literally the hope and, and, and relief in their face that they have this tool 24 seven. And also that I didn't have to give them my cell phone number for them to text these same questions to me. And by the way, I will tell you just as a, you know, I didn't mention this before, but I taught a similar private class, um, like three years ago. And the guys who set up the class were texting me and then for like a year they were texting me questions about FileMaker and I'm happy to do that but then the next year I came back to do a follow-up class and I revealed

to them what I had done is actually taken every question they asked me about FileMaker in the text message thread and fed it into ChatGPT and then responded to them so what they realized is for about a year they were actually getting guidance that they thought was from me but from ChatGPT and they believed it so that's our Turing test on whether or not you can use these tools so the other part we'll say is that we're recording this on friday june 7th or something like that little peek behind the curtain but i say that because on monday do not forget wwdc it's 10 o'clock uh am on the pacific coast i definitely think that people in our community should be tuning in for that uh ronnie's not gonna make any comments but he agrees uh with that too uh and we in our next uh podcast will we will likely be alluding to not only more discussion on the new Claris AI topics, but maybe how there's some Delta with some of the stuff that we're going to hear from Apple on Monday as well, too.

So definitely tune in for that on Monday. Awesome. So much happening so fast and I'm loving all of it. All right, Ronnie, thanks again so much for joining. And one last tease. One more plug? No, we'll see Ronnie again in July. Sorry. We're going to come back and have some fun with Ronnie next time around. So you guys will see Ronnie again. Ronnie, thanks so much for showing up here. It'll be nerdy and awesome. Thank you so much for having me. It's fantastic. Thanks, everyone. See you guys.

Back to top ↑

Have something to share?

Join me for a conversation on ClarisTalk AI.

Schedule a recording →