navarre.aiClarisTalk AI · the FileMaker AI podcast
More talk about Agents with Ian Jempson
Matt and Ian talk about running local models, the emergence of Agents, different models for Semantic Search, upcoming conferences, and more.
Apple PodcastsSpotifyYouTubeAll episodes
Transcript
Transcribed by machine (whisper, large model), 10,730 words. Names and product terms are sometimes misheard; the audio is the record.
Claris Talk AI. Welcome to Claris Talk AI. We have special guest Ian Jimson. Welcome, sir. Hi. It's good to be here. So I've met you a few times at different conferences, and you've got a much deeper background than most of us in the FileMaker community with AI, because you have several years before FileMaker, before we all knew it was a thing, you were working with it. Can you tell me a little bit about that? Yeah, I guess I'll start off saying I've been working with FileMaker kind of forever. And about 12 years ago, I kind of moved away from the FileMaker world and worked as a software architect in a large kind of marketing company where I basically looked at architecting sort of large data flows from banks to public their communications.
But as part of that, what I ended up doing was going back to school and doing an MBA. And while I was doing my MBA, I spent a lot of time looking at how AI fit into sort of the strategic landscape and how it was affecting business strategy. So that's what I wrote my dissertation on and came up with a couple of business ideas around that. So when I was finished that, I left my job as a software architect to start transforming digital. Basically spent a lot of time working on products around extracting, looking at the data, looking at what's happening in the world, looking at what's going on in the world, looking at what's happening in the world, looking at what's happening in the industry, looking at what's going on in the industry, looking at what's happening in the industry, looking at what's happening in the industry, looking at what's happening in the industry, looking at what's happening in the industry, looking at what's happening in the industry, looking at what's happening in the industry, looking at what's happening in the industry, looking at what's happening in the industry, looking at what's happening in the industry, looking at what's happening in the industry, looking at what's happening in the industry, looking at what's happening in the industry, looking at what's happening in the industry, looking at what's happening in the industry, - Yeah, and the community is very glad of that.
I'm glad of it. 'Cause I've learned a lot from taking apart your example files and looking through your slides and stuff like that. I'm definitely in the group of developers that has sort of been playing catch up and I've just fallen in love with it, I think, through the last couple of years. So, you're presenting at Engage coming up, right? - That's right, towards the end of March. Engage in Austin, in Texas. So, I'm really looking, looking forward to that. I'm going to be speaking again about semantic search. I've given a number of presentations on this already, but this is going to be informed a bit more by some of the conversations I've had with people who've tried to implement it in the real world and sort of what are some of the things that they found difficult or tricky.
There's some, Claris has provided some really, really good training materials on how to, for the certification for that. And I was really pleasantly surprised by them. They're very, very good. So, I recommend those to anyone. But I'm also going to be talking, like I say, about some of the issues that people have been having about, like, how do I apply this? Because people are saying it sounds like a great feature, but like, where does it fit into my solution? You know, how can I actually use this in the real world? - I want to take a little nerdy aside here, having done a little work on semantic search. Now, after we talk about this, I want to go back to a session you just presented at Claris.
But I've been working a lot with semantic search, but I mostly have been working with a couple of open AI models, like Text 3 Small. What other models have you used and how, what kind of results are you getting with them? - Okay. What I've been doing is, generally, when I start playing around with something, I usually work with open AI, just because their APIs are really straightforward. They're well-documented. If you run into trouble, there's loads of places to look for help. - Including ChatGPT. - Absolutely, yeah, it understands those things. - Yeah. - It understands those things very well. Or perplexity.is another one that I use quite a bit for that type of thing.
But I also run a number of models locally. So, I'm just trying to remember the name of them, but there's a couple of different smaller embedding models that I use. So, I mostly run them in a product called OLAMA, which allows you to run large language models locally on your computer or your network. - That's like LM Studio, different, easier, better? - Absolutely. It's a similar kind of thing. - Okay. - Same kind of thing. I find it a little bit easier than LM Studio. - Okay. - But it's, LM Studio provides a bit more of an interface and showing you what's going on, whereas OLAMA tends to run in the background and you can interface it with it via an API that provides an open AI compatible API.
- Okay. - So, the code you wrote for open AI just works. - Oh, nice. - And that's the same with LM Studio as well. That they're both have implemented. Open AI compatible APIs for embeddings. So, the code you wrote for open AI will just work when you point at one of these. - And then you're not paying the price. And what's the performance like? - For things like embeddings, it's pretty good. It, I mean, it depends on the computer you're running it on. If you're running it on a Mac, say like it's a Mac with an M1, M2, M3, or M4 chip, you're gonna get pretty good performance because those all have GPUs built in. And LM Studio and OLAMA are really good about taking advantage of the GPU in your Mac.
If you run it, say inside Docker, a Docker container, which you can do on your Mac, you'll find performance isn't as good because Docker isn't able then to take advantage of the GPU. So, you're running it entirely on CPU. - Yeah, yeah, yeah. - Kind of math that these things are doing, they're much, much faster. They're much faster on something with a GPU. - Yeah, and I love AWS and a lot of the AWS instances, all of the instances I've been playing with are really only CPU heavy. So, I'm sure they'll have some GPU heavy or only ones, but I haven't heard of that yet. - They're expensive. When you look at, oh, how much is this costing me per hour? It's like, oh, wow, it mounts up.
- Yeah, for sure. So, you just presented at Clarus, you're a big company. - Yeah, yeah. - And you're a big company. - Yeah, yeah. - And you're a big company. - Yeah, yeah. - And you're a big company. - Yeah, yeah. - And you're a big company. - Yeah, yeah. - And you're a big company. - Yeah, yeah. - And you're a big company. - Yeah, yeah. - And you're a big company. - Yeah, yeah. - And you're a big company. - Yeah, yeah. - And you're a big company. - Yeah, yeah. - And you're a big company. - Yeah, yeah. - And you're a big company. - Yeah, yeah. - And you're a big company. - Yeah, yeah. - Yeah, so Apple's European HQ is in Battersea in a converted power station that's been renovated.
And if you remember the Pink Floyd album that's got like the power station with a pig flying over it? - Yep. - Animals? - Animal, I think so. - Yeah. - Anyway, that is Apple's European headquarters are in that power station, which has been converted to offices and extremely expensive residential buildings. - Mm-hmm. - And it's a fabulous location. Anyway, so digressing a bit, sorry. - I wanted to know, yeah. So Claris' headquarters is there too, isn't it? - Claris is there as well, yes. - Okay, okay. - So I presented on, and I was talking about building AI agents in FileMaker or integrating FileMaker with AI agents. So that's a term you're hearing thrown around a lot - Yep. - right now.
Like AI agents. And to be totally honest, most of what people are talking about with agents are kind of, it's like rebranded workflows, except they're now incorporating AI in some way. So a lot of them will talk to an LLM. So the demo I gave, I gave a couple of demos. One was the Anthropic desktop app on my Macintosh, talking to FileMaker. And I can actually show that if you'd like to see. - Yeah, I'd love that. Yeah, yeah, yeah. - So Anthropic has a desktop app kind of like the ChatGPT one, right? So you run it. - Yeah, exactly the same kind of thing. Anthropic have built what they call MCP, which is Model Context Protocol. And it's a framework that allows their systems to communicate with AI agents.
So I think they're trying to get ahead of things by they basically said, well, we've built this framework. This software framework. If we get a lot of people using it, then our framework is going to be kind of the go-to choice and people will use our systems more. So, but it doesn't only work with our systems. So if I can share my screen, if you bear with me one second. - Sure. - I will try to share the right screen, what desktop? - My favorite feature in Zoom, by the way, is if you go to the advanced tab for sharing, you can share a portion of a screen. It'll draw like a box that you can resize and move around. And then any windows from any application that you move inside that little window are shared.
It's great. Oh boy. So yeah, you can try to allow access here and hopefully you won't have to quit and relaunch Zoom. - Okay, can you? - Yeah, I see a Zoom. I see the webpage. - Can you see? - I'm not seeing FileMaker. I'm just seeing the Zoom webpages except cookies. I've shared the wrong screen. I'm sorry about that. I do that, I think, at least once a week. I don't know. You do it pretty regularly. Okay, so I want desktop. It's really hard with Teams, because with Teams it shows you all the windows and all your applications, and picking the right one is, whew. Actually, what they've done is I've noticed they've actually now just displayed numbers on the screens, which is handy.
Oh, yeah, there you go. Yep, now I'm looking at it in FileMaker Tasks. Okay, yeah. So what I basically did was I took the... So this is like the... I just created a new FileMaker database. I took their standard tasks database, which is just the basic template that comes with the system. I added a few tasks, got two... There's only three, but it's enough to prove the point. So we've got two of them that are in progress and one that is pending. So if I just run this... I'm just showing the recording rather than showing this live, because I've done enough demos to know that running them live never works. So I'm opening the Claw application, which is very similar to the ChatGPT app.
Similar kind of thing. You can see, how can I help you today? And so... Choose your model. Yeah. Yeah, exactly. What are my pending tasks? And you can see it's asking me, do I want to... Do I want to allow this tool to run? Because when you run the tools, it needs to get permission. So it's basically telling me... And that did an API call to FileMaker server? Yeah. Okay. Yeah, so basically I've written... What I've done is I've written the server behind the scenes in Python. And so it's basically the Python server is like in their framework will speak to FileMaker. So what you can see is actually it's quite nice. It's basically said, you know, what's in progress. It understands what I asked before.
Tell me I've got two tasks in progress and I've got three active tasks. So I thought that was pretty cool. It's just the beginning of this. It's like a very simple kind of integration, but that could be extended to include running FileMaker scripts or any number of things. Yeah, yeah, that's cool. So like it... I mean, you sort of... Well, this is one of the things, right? Is it deterministic? Do you have to determine in advance what it has access to? And I think to some degree, of course, we'll need to do that. Yeah, absolutely. It's very restricted in terms of what it can actually do in two ways. The code behind the tool, which actually I don't have handy on this computer, it only exposes certain functions to Claude.
So Claude is only aware of those functions that I have exposed in the code. Right. And then as a second backstop thing, if I just back up the video here, find the right spot, you can see that when it went to call the tool, it again, the Claude app is asking for permission to allow this tool. So that's because... Obviously, the computer security risk of letting these things run rampant on your computer. Yeah, definitely. Not just for FileMaker, but for whatever, right? Yeah. Absolutely. So this is something that really people need to be quite careful about. But you can see that the function was called get tasks and ask FileMaker was the name of the overall agent. Would this need a Python app?
Is there a way to configure it to just go directly to make an API call to FileMaker? I guess, how simple can this be made? No. Basically, the code needs to be written in this MCP framework they've got. So the example I chose was Python. I'm not 100% certain, but I'm pretty sure that they've probably by now released a JavaScript version of the framework. So you would need to write code in either Python or Python. Python or JavaScript to write the MCP server, the model context protocol server. Okay. So now the Cloud app isn't the only thing that could be a client to that server. You could also write servers in other languages, sort of clients in other languages, but they've built the ability to function as a client into the Cloud desktop app, just again, because they're trying to encourage use of their tools.
Yeah, that's cool. Yeah. So now the other thing I showed, let's kind of go in the other way, that this is using the Cloud app to run things. The other demo I showed was, and I believe you've seen this one before when I showed it in Malmo, but is we're using FileMaker to control things. And in this case, I don't think I'm sharing the audio. I'm sharing the audio output from this, but I'll just talk it through. Yeah, I'll just talk it through. So basically what I've built is the build. Oh, I do get the audio. Okay, great. I'm recording the audio directly into FileMaker. It's being transcribed by a local whisper model on my computer. It's now talking to a large language model.
And I'll just pause it. I'll just pause it there for a second. So the command I said was, I said, tell me which tools are available to me. And so the large language model running in Ollama, the model is actually Llama 3.1. It's basically taken information I've sent and it's identified three potential tool calls that it can use. It's predicting sales, viewing historic sales, and then tool availability description, which is, which is actually this script that is running right here. This alert is the one, the tool. Yeah. Meta. So, because obviously the LLM has no knowledge of what it can do in FileMaker unless I tell it. Sure. So I'll play that on a bit further. And I mean, presumably you could also have it, you know, make records and whatnot.
So anyway. Yeah, yeah, absolutely. In historic sales with sweatshirts. So interested in historic sales for sweatshirts. Basically after the LLM comes back and it's now run, basically a sales report showing me the sales for sweatshirts. Yeah. And a regular old FileMaker report. Cool. Sorry, carry on. And yeah, it's in a FileMaker found set. Yeah. Yeah, absolutely. Sweatshirts. I've asked for now for the green or red ones. I can't remember which color it was, but again, That's shown the green sweatshirt sales. Got it. This is interpreting a command and then sending, it must be sending some JSON to a script in FileMaker that says, go look in this table, go do this search, sort it by this, blah, blah, blah.
Okay. You've had to predefine it. Go ahead. There's one more little bit of the demo, then I'll show you what it's doing behind the scenes. Okay. So let's ask for the CEO salary now. Now, the thing is, it doesn't have a tool to tell me the CEO salary. Now, obviously, I've made this kind of jokey error message, but essentially, it can only do what we tell it it can do, and what the FileMaker scripting we're running will allow. So in terms of how that actually works, if I find, just hang on a second. I think it's tucked away on different screens. So this is the talk I gave at Claris, and what I'll do is I won't go through the whole talk, but if I go to this one, for example, this is the one I gave at Claris, and what I'll do is I won't go through the whole talk, but if I go to this one, for example, This is the one I gave at Claris, and what I'll do is I won't go through the whole talk, but if I go to this one, for example, This is the one I gave at Claris, and what I'll do is I won't go through the whole talk, but if I go to this one, for example, This is the one I gave at Claris, and what I'll do is I won't go through the whole talk, but if I go to this one, for example, This is the one I gave at Claris, and what I'll do is I won't go through the whole talk, but if I go to this one, for example, This kind of summarizes what we're doing.
We've got the user speaking. Basically, we're doing the recording. So the first part is just the top part here is just dealing with the transcription, and then it returns the transcribed text back to FileMaker into a web viewer. So it's basically the transcription is recording the audio through a web socket to an API that talks to Whisper, and it returns that value back to FileMaker, and FileMaker runs a script, which sends a JSON request to the large language model in Ollama. Right. You could also just type a request in to get back to that first part. Absolutely. It's not as cool, is it? That's definitely nowhere near as cool, yeah. So essentially, we're asking the LLM which tools can handle the user prompt.
So we need to tell it which model we want to use, which tools are available, and what did the user ask for. And so we need to tell it, again, which tools, but we also need to say, what parameters of those parameters are, which are mandatory. And basically, the tools in this case were just FileMaker scripts. So I've got one record per FileMaker script that I'm allowing the system to run. Sorry, there's a bit of a-- Oh, I see. Yeah, so you have data. You have a table that has FileMaker scripts with parameters in it. Okay, cool. Exactly. And this is basically what we're sending, the information we're sending. We've got the script. The description, the more descriptive this description is of what we actually want, the more accurate the LLM is going to be, because this is what it's relying on to decide what script to choose, if that makes sense.
Yeah. And then the parameters, some of them can be mandatory or not. And then basically, I'm bundling that up. So I've got a calc field to convert this to the correct JSON format. And so the prompt looks a bit like this. So we specify which model. And so it needs to be a model that supports tool calling. So LLM 3.1 does. Things like deep seek, for example, we could not use for this, because it's a reasoning model. It's not designed to return simply just JSON. When we're doing this, we want to get back just JSON. We don't want to get back big chunks of text, of stream of thought consciousness. Yeah, that totally makes sense. Okay, okay. So are there, for example, are there OpenAI models that I would maybe know that do that?
So I will. Are there, well, like OpenAI models, I mean, 4.0, I don't think can do that, right? I believe it can, yeah. Oh, okay, cool. If you specify, if you tell it format JSON, a huge amount of work. Oh, well, of course it does. Like if you make an API call, it always comes back JSON. Yeah. Yeah. Okay, sorry. But that's different than, yeah, different than. Okay, okay, continue. It's really, really important, that aspect. And that's something that a lot of work went into over the last year with large language models, because people want to interact with them programmatically. And basically, if it gives you back like long chunks of text, you can't do that. So you used to have these, you'd see sample code that was interacting with LLMs that would basically say, give me the format in JSON.
It would tell it. This in several different ways, trying to force it into giving back JSON. And then it would receive the response. And then you'd have a whole bunch of code trying to parse out everything that was not JSON and get rid of it. So it's stripping out the text saying, yeah, I certainly I'd like to help you, you know, that kind of thing. So they're really good about it now that some of the models have been designed to support tools. And so they're very good about returning just JSON. Well, we don't want to stream the response. We tell it. And then we've got a system part of the prompt, which is got two parts. We've got sort of the tools that we're sending the parameters.
And it looks, this is the open AI example. You can see that we've got the message saying what's the weather in Paris. Then we've got the tools. What type is it? It's a function. The name, the description, then the parameters, all that kind of stuff. So what we do is we basically build this up, except instead of giving it Python functions, we give it FileMaker script. So then what we get back is we get the response to our request from the LLM into FileMaker. And the response looks like this. It says function name. This is one example with no parameters. Mm-hmm. But for the other one, to view the historic sales one, it's told me the script name, view historic sales, and then I've got the parameters that I need to run that script.
And then at that point, we use perform script by name. Yep. To run that function. So it's kind of like a conversation I had with Wim de Kort probably a couple of weeks back, where basically it more or less was saying like, well, it's really, this is all API calls. Yeah, exactly. And FileMaker gives us all the tools to deal with API calls. With insert from URL and the ability to create and parse JSON. So FileMaker gives us all the tools we need to do this kind of interaction, which is really cool. I should say at this point that in future versions of FileMaker, some of this stuff is going to become easier. Yeah, I mean... The more advanced tools to deal with this kind of thing.
Yeah, they're clearly putting resources in. I'm kind of thinking about this because I've been playing with something that kind of does something a little bit like this, but it shows you the results of your query in a, of your prompt, I should say, in a web viewer. But in order for that to work, you have to actually send it all of the FileMaker data. The thing that I love about this is you don't have to pre, you're not sending any FileMaker data. You're just sending and making it construct a command or something, if you will. Yeah. And then it's just doing a regular old FileMaker search using regular old FileMaker... Absolutely. ...tools, which actually could be, you know, they don't, they don't have to be regular file.
They could, you could do a semantic search or it could run it, run an API... Absolutely. You know, a generative API call. Yeah, anything. So this is, this is, what we're actually doing here is we're implementing one of the design patterns, like Anthropix, the AI company have got a really, really cool blog post on building AI agents. And... Yeah. ...they've identified, they said that building full on agents, like really try to avoid it. What you can do is you can build things that are a lot simpler to actually get the job done. And they've identified a number of design patterns, and this is one of them. So this is the, the, the, the routing pattern where you've got the input, your LLM call functions as a router.
And then what it does is it decides which, which call goes next. And that's basically what we're doing here with FileMaker, except instead of saying in place of, LLM call one, two, and three, we've got FileMaker scripts. Yep. Yeah. Now, if you're doing this, it's best to keep things relatively focused on a certain area or domain of knowledge. Because if you, if you pass it a big chunk of a hundred scripts or choices, it's probably going to get confused. Whereas if they're kind of related, what we could do is we could use the first LLM call to say, narrow down to the area. Let's say whether it was finance or marketing or whatever, and then ask, ask a specific, sorry, specific request related to, to that domain of knowledge.
And then you can have a much more focused, detailed prompt that gives much more context about that knowledge domain. If that makes sense. Yeah, it does. Cool. Let's see, what might I have missed in this? Yeah. Do you have to send, in the first call, at some call, do you have to actually send the list of all the FileMaker scripts that it can call and kind of what their AI is or is that, or what their parameters are? Or is that only... Yeah. Is that another, comes in another stage? Stacking is a different... That's, that's sort of early on. Okay. Sorry, I'm trying to find it. Yeah. I think you did see it. Yeah. Good showing. So in here, it, okay. So this is, this is a curl call, but this is what we're really doing behind the scenes with insert frames.
This is what we're doing behind the scenes. So we build up this chunk of data here as our JSON that we're going to send. Yeah, tools type function. We would just use this exact same format, but the function name would be the name of our script. The parameters would be the, our script parameters that we got from, from here. Got it. So in the tools would be an array. It's another example. Yeah. So the tools is an array of all the records in the, in the table with the names of the script. Okay. Okay. That's correct. Okay. Nice. Like I was saying that I think what we probably would want to do would be to try to make sure that it's fairly focused by what, by business domain or area of knowledge, and then you'll get better results.
So the user can improve the performance of this in a couple of different ways. They could go into this FileMaker folder of FileMaker table and change the, the description and, you know, like the description field, I think you said is really important. Yes. And put better parameters in for that. Yeah. They can add, add more tools, right? So that automatically then gets into the script to deal with it. And then also choose a different model. So when a different model comes out, it'll do a better job understanding stuff. Yeah, absolutely. And I just had something interesting I've been looking at that because what you've got is you've got different models that you may want to use in different circumstances.
So one of the things I'm looking at is. Basically having something that is like a first query that you would pass the query and then it would basically give you the answer. Do I need to run this on a big model or can I just run it on a small model? Because small models are cheap. Big models can get expensive. Yep. And you don't want to run everything. You don't want to send everything to OpenAI's most expensive model. Yeah. If you can get large or whatever, you know, some, some tiny cheap model. So if you're, if you're generating. If you're generating a SQL code, for example, there's a lot of smaller models that are actually now really, really good at generating SQL code.
So you probably wouldn't want to send that to OpenAI's most expensive model. Right. Or Anthropix's most expensive model. You want to do that as cheaply as possible. When I really showed a demo that actually that's what it did. So in your case, you're actually having the code run file, micro scripts to do presumably, you know, one of the things I like to call the magic quartet, go to layout, enter, find mode, set field. Perform fine to get, to get the phone set back, but it could, but it could also do a SQL call and get keys. And then you use other methods and FileMaker to show the, show the data. Okay. Exactly. Yeah. Okay, cool. Yeah. I love that. I love what this leverages in FileMaker.
Right. It's, it's just kind of like there's, there's so many different options of how to do that. The thing is that some of the basic stuff in FileMaker we take for granted actually is just like what you've described there, you know, to just performing a find is so, so powerful and, and what we can do is we can use, we can now use LLMs to decide which things should we be doing. So yeah, it's, I'm just really, really excited by what we are going to be able to do with all this stuff. Yeah, for sure. I mean, and requests or requests emit after you do the fine sculpt it in some way, loop through records, go do a fine. And then go to a related for the entire set, you know, add or join, whatever.
All that stuff is just so easy to code in FileMaker script. Yeah. So what, what are some of the real world use cases you think that that flow or do you have, you have anything in production for clients that use something like that? You think that? No, no, I don't. That to be honest is that looks kind of, it looks kind of cutting edge. Yeah, yeah, yeah, yeah, yeah. If, if, if, if one of your listeners has something that they would like to like to do give me a yell and I would love to help them, you know, put, put that into production. But I think there's going to be the closest thing I've got is, and this is something that's basically being tested at the moment is it, it is a reporting system where somebody can just enter plain text queries and it will actually, in this case, it does use SQL to, to query a system.
So but the other thing it does is when it generates those queries and it gives back the results, it, it passes it off to an LLM. And so they get the results back in, in plain text if they want. But the other thing we could do here is have different tools. So we could give a tool that would return it in plain text. We could have one that would give them back a chart, an Excel file. Yeah. JavaScript HTML, all these different, different formats could all be implemented as simple, simple tools. And so it'd be very extensible. But the other thing we've done there is that what it allows them to do is to say, this query actually gave me really, really good results.
And so they can just tag that. And then that can be kind of almost favorited. So in that case, it doesn't need to go to the LLM anymore to query, because we know we've got like a SQL query that gave somebody really meaningful, useful results. And they can just run that query directly. So that kind of cuts out the LLM. Yeah. I like that. In terms of exploration to find new, so when someone's exploring data, this gives them another tool that they can, because a lot of people who are, if you've got a company manager who wants to explore the data of their results, they don't want to learn how, they don't want to learn Python or R or scripting. They just want to sit there and look at the data and ask it, ask questions of it.
And this can give them that capability. Well, I mean, if they did learn something like R, really hard. They'd still have to know kind of exactly what they want and there's no, you don't get all that stuff for free. You have to do a lot of stuff in advance. You could also just use FileMaker for that, right? Like here's my tape, here's my big, you know, leaf node table with all the line items in it. Yeah, yeah. You know, do a query, slice and dice, sort by this, you know, go to show related, whatever, things like that, which you can get a lot of information. But this is so much more natural and so much faster that it really gives you a field of a capability of knowledge.
You've never really had in FileMaker, which is this, you know, interaction or conversation with your data. Yeah, because LLMs are, they're, they're stunningly good at generating SQL. They're also really good at generating Python code. So you can, some of these things behind the scenes of a query like that, a lot of stuff can be going on to query the data. And if you pass the data structure to the LLM, frankly, the LLM is going to be able to understand the. Behind the scenes data structure much more than, because if you think about it as database developers, we, we need to understand the business problem. We understand the business problem, but we also enabled, we also have to translate that into behind the scenes generally relational databases.
And if you think about it, a lot of the relational database structure, much of the structure is there to support the needs of the database rather than. The needs of the user, a senior manager at a company who wants to know about the, you know, the green sweatshirt sales last month, doesn't want to learn, doesn't care about normalization or shouldn't, shouldn't need to know about it. Right. The LLM. Easier if it was a huge flat thing, right. For them, for their perspective. Absolutely. Yeah. And the LLMs can take away a lot of that kind of hassle of the query. Yeah. They can, they can make. Just, and the other thing is that someone can sit there and they can just try it and try it and try a whole new variant.
And it takes away the loop of them having to talk to a developer who sometimes for all the best will in the world, sometimes we can be a bit stroppy and difficult to deal with. And you get a really quick turnaround on looking at data. Right. So it's not, so this is not a, a, a workflow, not a, what am I trying to say? A job that a FileMaker user is really ever used to doing. In FileMaker, you don't experiment search, search, search, search, and try to get results you want. You kind of know what you want, you're going to get it and you don't keep trying. But Google, for example, like if you search for results, you don't find it. You try to keep editing your query.
So people are sort of trained by having to outthink a little bit like the prompt to get the results you want. And I think that's good. I think that's a skill. I think that's a skill that our users have. They just have never been able to use it in FileMaker before. Yes. Yeah, exactly. Maybe like searching for something like a name. You got to try to spell it different ways. Enter semantic search, you know? Yeah. So, yeah, I mean, actually one of the examples that I've got in my, my slide deck for Engage is around, it's around semantic search and the contextual search. And I basically went to Netflix and I type into Netflix, I'm looking for a film with a strong female lead and, you know, I'm, I'm expecting, I'm going to get back, you know, alien, alien.
You know, stuff like this, what I get back from Netflix is just basically a database message saying like, there's no indexed fields of the cuff. Yeah. It doesn't have, uh, and, and yeah, to me, it just seems like that's such an obvious kind of use case where you can make it much better and more meaningful. Need to worry about what what's indexed or true. In, in the examples. That I've been doing, um, semantic search is not supposed to really work very well on names because they don't really have meaning people's names, but the, and then if you just use a model like text three small, I think it's the name and you index, uh, and you embed nothing except the name of the person you could search for something like Swedish names, literally that's the prompt you can put it, not a person's name.
And it will kind of give you names that are Swedish that are common. Um, or if you type in a name, that's that is Swedish. Um, you know, like, uh, B or something like that. And we were just looking at them before that call. Um, uh, then it will show you names of that variant, but it will also show you other names that are not close to that, but are Swedish. And it works for any other language as well. Um, anyone that are supporting that particular model is, is multilingual, but it doesn't do everything. Um, so like the example that I did is I translated my name to a bunch of different languages. And then if you search for it spelled correctly or not, it comes up and you could search for it in any language and you get results in any language.
It's magic, uh, in any kind of a multilingual environment and even not in multilingual, because it's still correct spelling and it does a ton of other things. The results aren't in the order that you predict, but you get the result you want in the top 10 or 20. Yeah. The, the ordering of the results is yeah. In that case, I guess it's going to be interesting because ball maker tries to sort by some semantic similarity and, um, yeah, in that case where there's not a lot of context, I'm not sure how that really works. Well, file maker is not calculating the semantic similarity. The model is kind of doing that, but it is either is, or is not, you know, a match.
The model basically is what the limitation is there. I think. Yeah, that, that is to raise a sort of an interesting point because I was in a, there's a, um, a user group online where somebody had been having some issues with, um, semantic search and it wasn't quite working the way they were expecting. And, um, what I found interesting was when I looked into it a bit deeper was the differences in the results that you would get from different models, different models will give you different results. Um, and actually that's a place where, um, you can also fine tune embedding models to give you, um, that's not something people talk about a lot because yeah, there is that feature when you bet, I don't know how to use it, but I know there's a feature in there. I haven't played with it. I should, I should say, um, when you do the embedding, there's that one extra area we can put configuration.
Yeah, this is actually changing the model itself. So if you've got a, if you've got a very narrow area of domain knowledge, you can fine tune an embedding model to, um, this is not a, this is not a trivial undertaking. It's a fairly major undertaking, but it is possible that let's say you were working in finance that you might actually, then get a fine tuned finance embedding model. That's going to be, give you better search results and similarity results for finance. So, um, if you think of let's take, for example, again, with finance, but if you've got with finance, you're going to, they might be dealing with securities in the sense of, you know, what, uh, stocks and bonds, but they may also have security like computer security. And the word security has got like two totally different contexts and a standard semantic search on the word security is going to give you quite unpredictable results in terms of which of those contexts it's going to give you because it just doesn't know.
So that's, that's the case where, and that's why some people, when they've got very short sentences find that this stuff seems to give almost random results. Whereas if you give it much longer sentences, then you've got more context for it to know what to search for. Do you, have you used, um, I should ask a couple more nerd questions in here. Do you, do you, um, do something like Jason LD or another mechanism like that to put in metadata with the data that you're embedding? So, you know, in other words, rather than just a name, you're embedding a big, a Jason blob. Yeah. That says this is name data. The language for this particular name is this, the, um, uh, you know, the, the context of the name is this, the type of company they work for is that whatever. Um, and you're, so you're putting more than just the name in there.
And then when you do your prompt, you're embedding your prompt in the same way. And then, and so then it's actually doing, you know, the embedding, it has a lot more information than just John Smith to John Smith. Yeah. Um, I haven't done that with FileMaker. Um, but the, the system that I was using before FileMaker had it is, uh, something called pine cone. Um, basically it's a vector database in the cloud and that's what I was using for embedding before FileMaker had it. I actually have done that with FileMaker and the results are way better. Yeah. It's okay. And then you can also get much more information too. You could, like, if you're looking for, uh, if you're looking for like, uh, a product description or something like that, well, maybe that's, I'm trying to think of an example, um, uh, like, uh, oh, a good example would be like an invoice with its line items.
And a lot of times people want to say, like, I want to try to, rather than creating a whole invoice with all of its stuff, I want to find some customer who has bought this configuration before. And I want to find the one that's the closest to this one. And I have no idea how far in the past it is, which then I can start from as a duplicate. I've had this, I have this particular question a bunch of times. And so you can embed a JSON that says, this is a mechanical engineering project. These are the line items. This is the total price, um, you know, and then whatever other data you have, even text data. So essentially what you're doing is you're kind of converting that, um, tabular data into, uh, into textual, something that the model can work with. Yeah, into JSON with really explicit tags. So the tags, uh, are really explicit tags.
Right. So the tags, uh, are really explicit tags. So the tags, uh, are really explicit tags. So the tags, uh, are really explicit tags. So the tags, uh, are really explicit tags. So the tags, uh, you know, they, they, they really give you a lot of information. Um, this is stuff I'll be teaching in my classes coming up, uh, uh, on, on AI integration. I love it. All of us who are doing this have such a different approach. You're miles ahead of me. You've tried some things that I have not even had even done, but I love doing my own experiments and learning. So, well, actually, to be honest, I'm just, when you were just describing there, I was like, ah, that's what I can, I can try that out this afternoon. It works really well. And there's a thing called JSON-LD, which, um, take a look at it. It's, it's a specific format of it. And then it uses a schema that you send. Yeah. Um, uh, yeah. So J I mean, Jason L is, um, is yeah, that is a format that's used in terms of training data for models. And, um, that's a whole different. Oh yeah. Maybe that's the same kind of a thing. And maybe it's the same thing, right? Where you're here. It's essentially like a line of Jason by itself. Um, so if you've got a Jason L file, it's just like a whole load of lines of Jason, different Jason documents rather than one. So that's not what I was thinking then, but this, it sounds like a variant of what you've described. Sounds like a variant of it. I think so. Yeah. It uses like schema.org and you're putting a specific, it
actually has a reference to like a standard description for different types of things. And then because Jason has named value pairs. Yeah. Um, you know, you have company name, your, your name for that thing could be something, something in your industry, like mechanical engineering vendor name is this, you know, what you said there that with schema.org, that kind of goes back to like, um, with the tool calling and the structured data, because you need to get the results back in a highly structured Jason format that Jason format can't vary from time to time. It needs to be the same every time, a hundred percent of the time. So you can rely on it to parse it correctly. Yeah. Or every time you change it, you have to re-embed the data. Yeah. Yeah. Okay. Okay. A couple of nerd questions. Uh, I still don't know the answer of, um, whether there's a performance difference if you store the embedding container in an external container or it has to be locally in the file. Have you, do you know? Yeah. Um, there is a performance difference. Um, cause I, I actually chatted earlier on when they were building this, um, I w I was lucky enough to sort of get early access
to it. And I, I actually asked that very question to, um, one of the developers and they said, yes, it is slower that there is a performance penalty, but it's not, if it's externally stored. Oh, not very much. Okay. Okay. So it's not like a five times performance difference or something. No, it's like, it's like 5% or something. Oh, okay. Great. Great. Great. I've got notes somewhere, but it's all it's on that order of magnitude. And because the, that brings up, that also ties into another really, really important point in that, in that database backups, that if you've got these embedding, these vector embeddings can be very large. Yeah. And if you internally within your main database, you're now creating a backup issue and how do you manage that? So having them externally allows you to, it gives you more flexibility in determining your backups strategy to, to avoid backing up hugely. Yeah. Yeah.
Yeah. Now, oh, what they did say was that where there is a performance impact, a bigger performance impact is if you encrypt those, if those are insecure, that would make a big difference. Then they've got to work at, they've got to deal with that. Yeah. Yeah. All that apparently is more of a, um, there's more of a performance impact there. So it's something that's probably actually do a bunch of tests on to try to get some like hard numbers. Okay. Another nerd question. I think we're getting close to wrapping here. Okay. Sometimes when I do a search, like I like to play with thousands of records, sometimes I'll do, I'll do like three or four in a row and they're instantaneous, boom, boom, boom, or not instantaneous, but like a second. And then I'll do one and it will be like, oh, searching on, you know, 10,000 records and it will go, it will take a minute or a couple of minutes.
And I don't know why that is or how to control it. If you're searching, I'm not sure. I'm not sure. I'm not sure. I'm not sure. I'm not sure. I'm not sure. I'm not sure. I'm not tested this for a long time, but I, I did do some testing on this really early on that if you search directly on the context of what you're searching for, things seem to be fine. If you search through a relationship, it looked like things needed to get dragged down from the server, like every time. Yeah, no, this is still in a local table. So this is like a flat table. Yeah. Table. And sometimes it's fast and sometimes it's not. And this particular example, the server's really small, but when I look at the server stats, it doesn't seem like the server's doing any other work. I'm not sure where the work is being done, honestly. Do you know?
Yeah, it's, it could, I mean, to be honest, it might be something like the database cache, the local database cache on your computer decided that I'm too different. And now I need to just go and download everything locally again. Right. Okay. That would make sense because it's actually caching all the container data. Yeah. Yeah. Yeah. Yeah. So when you're searching on all these things. Oh, that's huge if that's the case. And what you do want to do is you do want to, if you can, kind of pre-filter your searches. So if you're able to do index field searches to reduce the number of records that you're doing semantic search on, things will definitely be quicker.
Yeah. I've definitely heard that. Sometimes, of course, you can't do that. Like if you have a big name database and you have 20,000 names, you can't really do a, you know, I mean, there may be ways to do it. You can say only look at the ones that have actually done business with us in the last year or something like that. I've got some charts on the performance and it's actually the performance kind of scales linearly with the number of records. Obviously. Yeah. For the search. Yeah. The embedding is different until you reach a point, a certain point. And then there's like a huge drop off. But that point I found was, I mean, I was testing this on a very small, like 16 gigabyte server. And that point I think was around about three quarters of a million records.
Oh, okay. That's a lot. Yeah. But by the time you reached there, I mean, you're now looking at a couple of, you know, several minutes for the search, but what happened was at three quarters of a million, it kind of went from just picking the numbers at random here, but it seemed to go from say three minutes to six minutes, like it was a giant. And then it went back to kind of linear again. So my My suspicion was it was something server-related. The server was just running out of memory or something like that. I can't imagine that it has to cache all the containers. Because, I mean, a container, like for a text embedding, for a text-free small, it's 12K.
Every embedding file is 12K. And if you have 10,000 records, that's a lot. That's a couple hundred megs. And it's not going to download a couple hundred megs to your cache upon opening the file because you would notice that. And when you do the very first search, it doesn't take any time because you would notice that. So there must be something else going on. I should have asked Ronnie. Yeah. All right. Yeah. I'm not sure. If anybody else knows the answer, definitely. Oh, absolutely. Yeah. Tell me. What else you got? What else have I got? So what have we talked about? Talked about. We talked a little. We talked a little about local LLMs, didn't we? But not. A little, but not that much.
Not that much. Yeah. So, yeah. When I was, after I did my talk at Clarisse, I was talking to a number of other developers. And one of the things that people were a bit concerned by was the need to send all the data to OpenAI or Anthropic or whichever cloud provider. And I use a product called Ollama. It's Ollama.com, I think. And it provides like a downloadable installer that will allow you to run a large language model on your local computer. Obviously, it's going to run much better on a computer with a GPU. So like an M series Mac, where it'll take advantage of that. But it's very easy to install, administer, and lets you run local models. It provides APIs as well.
So you could use it for, if you're doing a vector embedding with FileMaker, you can use Ollama to serve the vector embedding. You'd need to choose a model that's supported. But there's a number of them that will work with that. And the different LLMs. And it's quite good. The amount of memory you've got in your computer, obviously, is a huge determinant of what kind of model you can use. But most of the models, like an 8 billion. 8 billion parameter model, you'd be able to run on a regular desktop computer with no problems. 16 gigs of RAM kind of thing. Yeah, exactly. I mean, on that, you would not be able to run a 70B model, which has got 70 billion parameters, because that would require...
You might even struggle to run that in 64 gigabytes of RAM. Yeah. So they're not as powerful as, say, the models from OpenAI or Anthropic. Sure. However, they can, for a lot of purposes, they're more than good enough. And that example of the tool calling I gave was all running locally. It did not talk to a cloud service at all. Since the actual find doesn't have to make an API call unless you embed your find, which then allows you to do a vector search rather than a natural language search. Yeah. Yeah. So that's, I guess, one other problem to solve. Right. Embedding is... Well, I guess you also really kind of embed as you go. Like, every time a record is updated, there needs to be some sort of a way to re-embed it.
Right? Like, in my name example, you correct... You change the spelling of a name or add a new record. You're not going to use embed found set. You're going to embed that one record server side or whatever. Yeah. And if you do them kind of one at a time, there's not a massive load. No, it's fast. If you do need to do a large number of them using the insert embed... The embedding in found set can do it more quickly, but it also gives you several parameters that you can adjust, and some of those parameters can make things faster by changing the number of records you send at once. Claris has gone for, I think, what's a fairly conservative number, and I found that I could make the embeddings run sort of more than twice as fast by adjusting those parameters.
Oh, so it's that parameter section on the embedding of found set. That's the one area in the semantic... In the semantic world that you can actually specify the things in the script step itself that actually configures how things... Okay, cool, cool. I didn't know. I haven't played with that yet either, so I'll play with that, too. Yeah, I've got some slides somewhere on... Okay. I'll send them to you. Okay, great. Is that actually... Do you happen to know if it's documented in the Claris help files, too? Yeah. Okay, good. That's an easy way to do it. That is. Okay. I did some experimenting and sort of playing around with different values. So... I got some kind of stats on that as well.
Sort of the changes in performance you might see from playing with... And, yeah, so... Cool. Yeah, that occupied a surprising amount of time. I know. This whole thing is experimenting and learning, and I guess I'll leave you with kind of my one opinion on this. I think it's... I think us FileMaker developers, we FileMaker developers, have a really important task, and that is to use our imagination and really think about what our clients could use in AI. We're not waiting for them to tell us. We're using our experience and history of knowing what their job is, what they do, what they need, that they don't know that they need, because we've been doing that all along.
Yeah. You know, we watch over their shoulder, and we watch them, you know, print a file, and do some really workaround thing for PDF, and then emailing it, and we go, you know, there's one script step. Yeah, so things like that, I think. But I think for the AI world, it's a much bigger task that we have, but I think a really important one. So I think we really all need to build our skills and what the capabilities are, what really good use cases have been laid down, and we can repeat them and make a case to put these things into our customers. We're not just putting stuff in willy-nilly, because I think that's a recipe for failure. But I think searching for really useful use cases, from obvious ones like generative model to translate something from one language to another, to more subtle ones that, like what you were showing, that have a stack of things, right?
Like, it does a generative, which does a semantic search, which goes out to something, which then calls a script, which does a, you know, that's a stack of things. That can solve a really significant problem that the customer does not know they have, or create a solution that they could really benefit from that they don't know that they need, you know? I mean, one example of that sort of stack would be, let's say that you've got a web form where you get new contacts coming in. You can basically receive the web form, the contact information from the client. Yeah. In the background, the system basically can basically then go off, it could actually do a bit of research on the client.
You know, find information and come back. That's so true, yeah. And based on that, basically say, well, you know, this guy's asking for, this guy's asking you to rebuild Facebook, but, you know, the company's two people in, you know, the middle of nowhere. Right. You know, alternatively, it's like, you know, this is, you know, a multinational who's got a significant, and those kinds of things, you want to respond in different ways. Yeah. Oh, that's interesting. It's not the kind of thing that some of these things can be. Can be automated to help. Yeah, that's a solid use case. And I hadn't thought about that, but you can do an API call and send data and say, go out on the internet and search this information.
Yeah. Yeah. That's cool. Yeah, cool. All right. Well, there's a ton more to talk about, and we'll keep, I'll have you on again. Okay. That'd be great. And we'll keep nerding out about it. Yeah. Thanks so much for your time. This has been great. Yeah, it has. Just, I'm going to say one last thing. Yes, yes, yes. If anybody else is going to be in Austin, Texas for Engage, you know, it'd be great to chat about this stuff. Are you going to be going to other European conferences too, like the one in Spain? I'm not sure yet. Actually, they just announced Engage EU is going to be in Antwerp this year. Yes, November. In November, which I'm looking forward to. Definitely be there.
That's the one. If you're going to go to one in Europe, that's the one to go to, I think. Yeah, I would agree. There's other ones that are great as well. Rome Filemaker Week, which will not be in Rome this year because there's a huge festival going on there. Yes, because Rome is full. There is the one in Austria, Vienna, which is kind of the same week as .fmp used to be. Oh, yeah, yeah, yeah. Is that in June, I think? I can't remember. That's the first week of June. There you go. Yeah. Okay. Awesome. Thanks so much for your time. Okay. You're very welcome. It's been great. Thank you.