ClarisTalk AI · the FileMaker AI podcast
Claris Talk AI - A new podcast from Matt Navarre and Cris Ippolite
Matt and Cris kick things off with a new podcast that will mix discussion about Claris & FileMaker with a strong emphasis on AI. Learn what you can do with ChatGPT, ML, and other AI models, and how much you can get from InsertURL().
Apple PodcastsSpotifyYouTubeAll episodes
Transcript
Transcribed by machine (whisper, large model), 12,378 words. Names and product terms are sometimes misheard; the audio is the record.
Claris Talk AI. Hey, Chris. Hey, Matt. I got an idea. What's that? Let's start a podcast, you and me. Ooh. I love it. What are you thinking? Call it, I don't know, Claris Talk AI? I'm in. All right, here we go. Oh, exciting. We have three episodes before this one, which are small interviews that I did when I was in Austin. You and I didn't even really talk when we were there, which is crazy. I think that's the way you can know the conference is big enough, is when people you really want to spend time with, you can't or didn't. You just don't even see them. Yeah, it's a great point. So it was really all about AI. The recordings, the conversations I had were just kind of filemaker people, a lot of people from Europe, and they weren't about AI.
So that's what you and I are going to talk about right now. I was in your, what was it, your 400th session? 50th. I know. But thank you. I know. You're just doing an age joke there. No, 50. Yeah, I was really proud of that. That wasn't an age joke. That was a compliment. Thank you, my friend. I appreciate that. Yeah, and thanks for coming to the session. I really like seeing your smiling face in the crowd there. And yeah, I was really excited to talk about AI to Claire's folks. It was something I'd been looking forward to for a long time. yeah and we finally got to get together to do that so i don't know a third of the sessions were about ai there i think something like that yeah i think we had um you know certainly the the big hits like the biggest hits were uh ronnie uh talking in the general well there was mention of it in the keynote and um and then ronnie of course did his like smash hit of a session right off the top and then um i i did mine that same day uh we had a couple other people uh do some sessions as well too and i think every well and i i'm not sure which ones you went to i didn't get to enough sessions i really wanted to desperately but i think all the ai ones were smash hit like as far as attendance where it was concerned you know i went to yours and only one
other ai one so those ones i'm going to watch the videos yes at the time of this recording we do not have access to videos. Literally the AI ones I missed are the first ones I'm going to watch and rewatch a bunch of other stuff too. So absolutely. Yeah. That's the first thing I'm going to do the second the links come out. And it's interesting, right? So right now FileMaker doesn't have any direct AI calls. I think there's a lot of people out there who think you can just connect FileMaker to the chat GPT API call and boom, you've got AI in your solution. and that's true isn't it well uh no i mean i mean there was a set there's a lot to i know you're just it's all going to be set up this whole talk is going to be set up i i know well so first of all um i think it's really important that we you know from just a understanding what if people are listening to this they're they're in the clairs community and they're curious about ai right let's face it and so the number one piece of advice i would give is about chat gpt that it should be absolutely be part of your life as an individual who's just walking around you know walking the earth and in your daily work routine as a matter of fact joris uh did a great session on you know ways you could use chachi bt to do that but chachi bt is not ai right bard i mean
it's a it's a flavor it is ai like uh a browser is the internet basically right like it's a it's a way to it's a vehicle to get there although it's not a safe for business uh vehicle to get there you shouldn't share any of your business data that way and you api version you api version you absolutely could that's exactly what i wanted to tee up don't use chanchi bt for business but use the api and and that's i think the other most important part is everything's just api calls so for the clairs folks that have ever made an api call or are just frankly aware that you can do that you're already halfway there and not only is it api calls but it's also just taking json that gets returned from these models and parsing it and filemaker does an amazing job of that so all day if you if you know those two things already you're kind of halfway there and i'm not sure how strong it is but there's an open ai connector in clarice connect so you can use that as well yes there yes that's another important part too kudos to uh clarice there there has been an OpenAI connector in there since last year.
I think it's being updated as at the time of this recording, it had not been updated for the new endpoints on the OpenAI side, because OpenAI changed to basically just a different structure of what your API calls look like. Those endpoints are really confusing. I had a really hard time finding the one I needed to use to make it work. Anyway. Like the messages and user and system and all that kind of thing. The area where there's like 47 different numbers you can use like which model do you want g3g you know oh we can absolutely talk about models as well too i think that's an important discussion um so yeah there's support within flows or and by the way um uh there's a custom you can create a custom connector for it as well too as a matter of fact i did a training day at engage on uh creating a custom connector and i used open ai the new version you know the new endpoints uh as my example so there's definitely ways to get in there or just like straight up raw make an api call just like any other api call where you're putting all your data into your curl request and you're sending that over as a payload that is really if that makes sense from a final perspective you've got all the tools you need if i think that's a i think that's a one that really everybody should be aware of if it's not a tool in your drawer you know someone who can do it and it's not really that hard to do um i'm gonna i'm gonna
step back and say and ask a question a dumb question i like those can you share custom connectors like if you made one can you share it to me no well you there's no formal like share icon or anything what you do is you go in whether it's published or you're still editing and you copy the it's basically a json document it's not basically it is a json document and so you copy the json put it into a text editor save it and then you send that to somebody got it that is currently how you share custom connectors that's pretty great actually yeah and if you want i will I can share my custom connector for the new OpenAI connector. I can share that with you if you'd like to.
I'll send that to you. Yeah, I would like that. I started playing with it. You know, you can only do so much stuff. And to get to Hello World wasn't so easy like it is with other things. Yeah, I think really kind of framing of what is Hello World in the first place, like what like what are we actually trying to do with a call to one of these ai models is probably a really interesting thing to unpack like um i have a position on how businesses can use ai but i also have a i think almost maybe contrasting position on how i personally can use it and i do like to use it as something that i use as a an alternative to search for starters i can't remember the last time i did like a google search for example and i think that has changed everything you know permanently it's also an incredible resource from a coding perspective like just and i look forward to throughout conversations with you on this unpacking what you can actually use like i think there's a lot of misconception about uh truth you're gonna hear me say that word a ton as we talk about ai like how truthful these models are and how truthful they need to be and And as a little teaser, I'm going to tell you, we take way more responsibility for the truth that comes out of the model than the model does.
I hope I'm teasing. No, that makes sense. And it's the same thing with using it as a file maker tool. Like I would say, for example, if maybe this makes sense, it's way better as a script refactoring tool than it is at a term you and I have talked about before, zero shot. It just means, so zero shot means you just ask a question you'd expect in that one answer that everything you need comes back like perfectly. That's not a reality. It's not a reality. So when we're talking about hello world, that's a zero shot. Like literally saying, hey, answer a question, you're doing a zero shot. But if you have a conversation and you're doing multiple calls, that's not right.
And also the magic is like, you know, you ever, you hear people talk about like, you are a helpful assistant you know there's this like system thing there's a line in there called system and and what i think most people do is they just put you are a helpful assistant that's kind of a nothing burger as far as like change it's not like it was programmed to be a not helpful non-assistant you know it's like that's what it already is a health yeah oh we'll talk about that we'll talk about syria well especially because we think new series gonna come someday too but but anyhow good so i'm gonna i'm gonna ask you two questions we're gonna get into those conversations love it so first of all this podcast is probably going to be like half regular filemaker stuff and half ai stuff and also you forgot to mention the other huge audience of the people who are in the ai community who are dying to get into filemaker oh yeah i mean there's got to be knocking the doors down yeah yeah there's at least tens yeah yeah well that was supposed to be funny um do you have a laugh track i know petrovsky actually had one of those the podcast we did yeah okay i'll just giggle along then yeah that's fine um okay two questions one let's talk about foundations like in your session there was really some stuff that a lot of people don't know that i didn't really know that has to be there and then two i want to get
a real world so a real world example of an ai integration you've done because i know you do quite a few of them um okay so we made a joke earlier about just using connecting you know the with an api call and just go straight into the gpt model and have a little text field in your database that gets back which that's not that's not what we do so how uh what would what are the foundations yeah the first so there's really a couple sides to look at this is first of all the uh the amount of information you're sending in your api call i think is a really important place to start so i think the the misconception is the thing you're sending is like the question for example or the command right it's not all about chat first of all like right you could absolutely just be sending a command like hey write a uh product description for me right that's not a question that you're getting an answer for it's a command but you are still getting a response and and sometimes you're asking a question as well too right well the thing is is that the models themselves have so much information, like just such a wide array of possible answers they could give. And in the transformer technology, which is what the modern day AI language models
were birthed from, is all about. It's really just a probabilistic completion algorithm. So literally all it is, is you ask the next question. Yeah. What's the next word? It's just like when you're typing in gmail or something like that it's literally that exact same thing but like times a million like times 479 param billion parameters plus plus it knows about all the things you've talked about before right so like it's not a one shot that's the fundamental like one of the huge fundamental differences which is why like in my little chat gpt i keep channels for each conversation and you definitely should and and within that conversation and this is so important i'm glad you brought this up within the conversation you're having you have memory right so it not only knows the command or the question that you just asked but it knows ones that preceded it so what's actually happening if you've had 10 questions and 10 answers in a conversation when you send the 11th question so to speak you're sending the 11th question and all 10 questions and answer pairs as well so that's where the context come from now there is a limit to that and this is It starts to really slow down after a couple hundred or so.
Yeah, well, here's why. And this is one of the foundational elements that I think is really important for everybody to know. And that's what's called the context window, right? Like we know this idea of context and FileMaker. And it took us all so long to get there to understand what that meant and how it was important. And so I don't want to knock that out of people's brains. But context means something different in AI. Yeah, definitely. In the case of the conversation, it means you keep doing questions, answers, questions, answers, and then your next one and you're sending the whole history of the conversation over, which is great, which is why you can do just to kind of be practical about it.
You can say who is Matt Navar and then it comes back, you know, historically significant genius and the clear space, all that kind of stuff. Right. And then you go, how old is he? And then it'll answer. It's not going to say how old is who like it knows already because you said. But so I think that's just a really important thing for people to understand. Yeah. So here's the deal. When you get into about 100 of those questions and answers, especially if the answer itself is like a 10 page, it's hard to get 10 pages like ChatGPT, but you could get a really long response. Eventually, you've got this window of memory. It's not like a physical window. You could kind of, you know, metaphorically think of the window inside your ChatGPT that you can scroll all the way to the top.
Right. The problem is there's a limit to it. And that's called a like a token windows or your context window, token window, whatever. Yeah. The reason people call it token window is because there's a each model has a limit to how much memory it has. So when these first came out a couple of years ago, and some of them are still this way, it had a 2000 token, 2096 token window. You know, they always have like a multiplier. It's like memory. It is memory, actually. And so what does that mean? Well, a token isn't a word because words could be supercalifragilisticexpialidocious or the word, thank you, nailed it. Antidisestablishmentarianism. Is that literally, I think that is the longest word, right?
It was for a time. I think it's a medical term now. And so obviously that in the word and, you know, can't be the same unit of measure because you're paying for this and it's memory, right? So what a token actually is, is like a syllable. So it's like if you say navar, I don't know, it's really going to be two tokens. Now, it's not it's not exactly syllables. There's a formula and it's all part of an algorithm. Yeah, but it turns every single thing into numbers. Exactly. Every word, every phrase, every sentence, everything. Yeah. And so what it's actually doing is is completing tokens, not completing words. So that's what that's how the algorithm works. And so therefore, when it's trained, it only has a limited amount of memory during the training process.
So the early models, like way back in the early years of language models in like three years ago, they were only like 2096 tokens could fit in a window. So what does that mean? If you put, I'll give you, I'll give you a practical example. We would, and this is what we're eventually getting to. We put so much information into the curl request, so much information. You saw the session. We're going to get to that here in a second. That at some point, you're exceeding that memory and it doesn't hear you anymore. As a matter of fact, there is a possibility that it just goes error and ignores everything you just sent over and then just answers its own thing.
So you have to be really conscious of it. Now, if we have the same conversation next year at this time, we might not even care about token windows. And here's why. They continue to get bigger. So three years ago, it was 2096. As a matter of fact, three years ago is 2096. Let me give you a little bit of a visual. You're looking at the NASDAQ token chart? I'm going to show. NASDAQ features. actually have a fun little uh so let's take a look at this so yeah so token windows some people are getting this by audio but it's also on youtube yeah this is how we're uh doing our shameless plug that you can get this on youtube but i'll narrate this basically so going back to about june of 2020 that's when gpt3 uh playground uh like uh preview access first came out to the public it was 2048 sorry i said 2096 before yeah um and gpt3 only had that much window if you scroll all the way down we won't bore everybody with this but when gpt4 turbo was released uh that's the latest incarnation of the gpt4 model the fast version that was november 2023 we're talking a a high end of 128 000 tokens and i'm excited to report that um there is actually something
exciting to report about google's gemini and that is that buried in all of this terrible failure of launch press that they've been getting recently. There's something really exciting, and that's their Gemini 1.5. It's like a pro model that has a million token window. So that means send it all. For those watching YouTube, I think that's February 2024. Yes. Not 23. Yeah. Thank you so much. A million. So that's crazy. So to go from 2048 up the range and basically doubling all of them to get to a million is gigantic. so i kind of think of it in terms of like if you have a conversation with someone you just met uh the the window that the token window is only back to the beginning of that but when i talk to you i can talk about the caffeine podcast and that's in our window because it goes back 20 some years ago to oh my god howell's coffee where we did a podcast recording after yeah we were we We were each only six years old at the time when we did that.
But no, you know what? What a great, honestly, Matt, what a great way to put that. That's exactly what it was. And so, like, if you said something to me about the caffeine podcast, and I was like, I don't know what you're talking about, Matt. You would go, what? What are you talking about? You were there. We talk about this all the time. That's when you know you've lost your window. And that's super important. For as long as context windows are a thing, I guess I should say. And then, of course, the conversations you and I have had over the years, most of that has been forgotten, right? But this doesn't forget. Oh, sorry. Yes, correct. It does not forget. With a million tokens.
So what could you do with a million tokens? It doesn't even have to be a conversation. Again, I don't want to have people think that AI communications are chat. You can literally send, you know, I was just being somewhat disparaging about this idea of zero shot where you just send a question. expect like all the perfect answer let's let's work backwards from there as to why you shouldn't expect the perfect answer and then we'll talk about why the token window comes into play well the first reason you shouldn't expect it is because you are actually talking to like a super smart you know data set it has all the possible answers and if all you're saying is what's the best place to do whatever you know whatever the question is i don't want to get too you know bogged down in the actual question itself how do you make really good crispy french fries i actually asked that one recently perfect well it it has a ton of different answers there's no one answer as a matter of fact the other thing that people should be aware of is you could ask uh how do you make really good crispy french fries and then open up another conversation and ask the same question and get a different answer like there's no like i see headlines all the time in my news feed that say like ai says this or whatever and it's like no your ai when you asked it said that and by the way you're kind of tipping your hand as to what you had said prior to that question based on the
answer that you got so so that's the thing you have complete control over what you're sending over there and so what my philosophy is and this is what we do for our customers and we've it's a result of a significant amount of time of trial and error so that's why i have a lot of conviction with it is send it everything send it everything about you who's asking why do you why are you asking what are you going to do with this information when you get it is there any other information you can send along that might be helpful as well too so at a really high level that i think is the number one most important foundational thing and so the context window to tie back to that and to put a bow on this is how much of the information you can send so we generally do like 16k ones for business we can send tons of information so let's connect that to like a real example. I did all this work in public health. So they've got tons of years of data of disease surveillance stuff. Who got what disease? What was the outcome? I mean, it's named, but you would do AI stuff, obviously de-identified. What year was it? What was their race gender ethnicity blah blah blah so you could that's a gigantic data set right uh all related
stuff like that so if you wanted to be able to have a conversation with that data you couldn't just do that with a plugin what are the foundations and elements that you would have to do in file maker to to get really meaningful results from that you hit on this in your session and i want to slow down and ask absolutely yeah so first of all where does the let's play a little thought experiment uh where does the where do the answers to your questions on that data where do the answers actually live is the expectation that they live in like the model like somehow the model knows all the answers of those questions or in your data i'm leading yeah no there's no there's no answers in the data right i mean like if you say uh something simple like is the i don't know talk about disease stuff right it what are the uh what's the arc of hepatitis c and um from 2010 to 2020 how is it you know something like that now you're now just to to really take this one little baby step at a time you were saying hey that actual answer doesn't live in the data you're saying there is no record that literally says the answer you know the arc of hepatitis was nor is there text in there that tells me the kind of thing that i'm asking for right there's the
If I, as a FileMaker developer, I could go write a report and something like that. But I'd have to know what I want. Yes. You would query the data, right? Right. You would go into FileMaker. You'd do a, you know, speaking Claris language, you would create a found set that includes the answers to, well, includes the data that you, the human, then can then derive into the human language answer to your question. I will lead you a little bit because you showed a document that basically said, these are the kind of answers that I would be able to get by querying the data? And then you showed another thing that said, these are the things that I'm not going to be asking about it.
Yes, correct. Talk about those and how much work that is to make them and where they go. Absolutely, yeah. I would say to provide the appropriate context to what roles those are playing, there's things that you're doing even before that. You might have seen in the session, and by the way, we'll, if we remember, we'll put the links to that session in the show notes and the you know the youtube uh when we well when we have access to it but even before that even before you talk about the data you have to talk about who you are and why you care about the answer me the user yes so i'm so do you remember as a chat gbt plus subscriber yeah yeah uh and i'm not even sure if this is plus because i don't know what's on the not plus actually now that i'm thinking about it but there was a feature that came out that really nobody seemed to talk about and it was called custom instructions does that ring a bell at all yes i've looked at it and and you can for those that are listening while walking your dog you can go into your app and you can look down in your uh little uh bottom left hand corner where your your account information is and your chat gbt and you can click on it and there's a setting called custom instructions and it's a window a text window where you can go in and say this is who i am like for example i don't have anything in there matt and here's why because
every time i'm interacting with chat gbt i could be taking on a different persona right i could be doing it for me chris or i could be doing it on behalf of the client project or whatever so i don't have anything in there but my wife but like i'm a 35 plus year old file maker developer yes you know who teaches online blah blah blah and like information like your resume kind of stuff right absolutely uh my wife who's a uh a special ed uh teacher like like actually doing good work out in the world yeah unlike the other part of the household here uh she only would use it for that one purpose right so she could go in and say look i'm a special ed instructor these are the ranges of the grades that i teach or the lexile levels like she can put in because every time she's then communicating then that information goes along with it so what's actually happening in that case is she could just type in give me a lesson plan for you know uh whatever she's doing like that day right and so what's actually getting sent over when she hits the send button is her question or command along with that big custom instruction paragraph that she wrote right every single time she sends something that gets sent over so what that's doing if you think about completion it's actually saying i'm this person here's my question now give me answer so already
hopefully it crystallizes for everyone that yeah that will give you a more appropriate answer to your question yeah if in fact you're asking a question then if you crystallize spelled like c-r-i-s-t-a sorry okay okay let's connect this to um like a what you would have to do as a file maker developer with your data set so so literally with you yeah before you would ask a so the the trick the kind of trick question that i asked you before was where does the answer actually live and you you actually gave it a great answer to it you said well the actual answer that i'm looking for the plain english this is what the you know the the text string answer is does not live in your database but the data does the data to answer that the arrays of data the all that information is in there so i'll go another step of the question does the text answer or even the data that you need to answer that question live in the model if now that's a maybe right like i think when you're talking about health services especially at a government level it for sure doesn't live in there because i mean i don't know maybe some of that's public record i don't know the nature of it that's probably useful to explore here that's well actually some of it is public record because you can compare like a lot of this stuff is published worldwide um so it
definitely it definitely can get information but then i have way more detailed data like in in my database right and and would you say that there's a chance depending on the type of data maybe not in this specific use case is there a chance that there could be outdated data in the model because when it was trained it was trained in october of 2021 and your data is up to the minute right so And or it doesn't have the answer at all, right? Like, for example, what if you were asking about a medical procedure that was just invented earlier this year and the model that you're asking the question of was trained last year? Yeah, so I asked it to search the internet for that thing and I can actually find it.
Well, sure, but I'm just going to say from a business use case, never search the internet. As a matter of fact, like just as a pro tip, you know, the GPTs that are out there on ChatGPT Plus, which are really cool. Developers went in and they were able to put a bunch of stuff in there. Don't use any of them that have web access to it because all that is just a wrapper to ChatGPT. I'm talking about like using ChatGPT for whatever and ask it to search the Internet. I know. I'm saying you're saying don't do that for business. Yes. For business. For business. Yes. As a pro tip, don't even do that. Don't even use GPTs in ChatGPT Plus that do that. And I'm saying because the author of those in the case of a GPT has no control over the noise that's coming in.
So they didn't really author anything. They just like pointed you to other people's stuff. In the case of business, that is introducing so much noise and so much data you don't have any control over. You'll never do that. Oh, that's true. And plus you can't do that in your API call as a data request anyways. Like when they built the model, they actually did selectively pick data, right? They selectively picked data sources, right? They picked Wikipedia, not Wookieepedia. Yeah, well, they did Common Crawl. So a separate institution from the actual model trainers actually collected the data. None of these model trainers collected any of the data. They just used publicly available Common Crawl data to do their training.
Let's go back to this. Yeah, I want to get the point across that your answer is either not in the model and it's training data, or if it is, you have no control over it and you don't want to rely on what that answer is to your question about the hepatitis trend. OK, so how can you make sure that you get the right the right answer that's good for you, Matt, or the user of your system with AI integration and you get the right answer? Well, we already talked about one of them. define who you are that's what i refer to as a user persona so it knows okay i can narrow down not only the way i'm responding to you but who you are and if i have to make a call as the model i'm talking as the model now on should i tell them this or that it'll be able to make those choices more accurately now the big thing is how does it know the right answer well you just told me earlier a person would sit down in front of file maker go search for it and create a found set the answer is in that found set of data right and then the human would have to interpret that into a sentence right so the trick is a hundred percent of the time you're using ai for business and you want to integrate data you're querying the data set coming back with the response
from your data set from your query you're including that data set with your question right in the payload so now we're talking about that token window right yeah exactly thank you so it's who is asking what are they asking and what is the answer right i mean it's it's not and then it's like well what is the language model doing well the language model is doing the part where the human interprets this and knows how to answer the question which is powerful extremely powerful role but what we're doing is we are dramatically minimizing its role in the truth equation here and i just want to linger on that for a moment and we'll get back to the formula here because that is so important if people if the two people that are listening to this right now take anything away from this conversation it's that you are in control of the truth not the model right so the 85 of headlines about ai right now that talk about hallucinations by the way did you know that the dictionary.com word of the year in 2023 was hallucination that's lovely so that means that majority of the focus is on what the model doesn't know right so why would you ask the model yeah yeah i get that so instead so the my metaphor i might have said this yeah use the model you ask the data for the truth and then the model does the interpretation i said it's um and this technique
is akin to a student taking an open book exam like how are you going to get the answer wrong i'm giving you the resource the answer is in there go figure it out right yeah so now we've got persona question and the actual data okay so what format are the persona and question how do i put those into my database or how do i put it into the straight well uh it's first of all it's straight text but it's uh the pro tips i would give is when you're writing those it's really important to like let the mob like take any ambiguity of interpretation out of it from the model so you would say something like um this is what my job role is and then like literally big you know in brackets then you put in the bracket of what you're it's kind of like defining like in a let function when you're defining variables and you're very specific about the the you know variable name equals and then you put what it is and then below you're using the variable it's the same kind of thing you're doing in these things so and i think some of your examples were like um you know summarize uh you know salmonella cases for 20 years by county yes and so those are you put it you put like what a hundred of those things in of different kinds of things that are real, that are samples of what the user would want.
Sure. I mean, whatever the most, whatever, whatever the, whatever contains the answer. Right. And so, so I think, so I think the thing that you're, you're thinking of is what I was referring to as few shot examples, like examples of what the answer looks like. And I know Ronnie mentioned this too, that is a separate thing from the data, the actual answers. So I just want to be clear. I know there might be a little confusion about it. And so far what we've talked about, we haven't gotten to the examples of what I want the output to look like or sound like or flow like. You're just straight up doing data, chunks of data. You're doing execute SQL against your own FileMaker database, and you're getting a found set of data and cramming that in there.
Or you're doing a DAPI call through an API to your FileMaker database, and you're taking the results of your query and cramming that in there. Or you're doing a vector database search, which we'll talk about later because I don't want to get on a tangent on that technology. Or you're doing an API call to a completely different source that has data in it and you're getting the answer. You're putting the answers in there and they can be non sequiturs, Matt. Like you don't have to write them. They don't even have to make sense. It can just be a bunch of garbage and somewhere in there is the answer. And the models have the intelligence and a level of reasoning to be able to interpret that gobbledygook into the correct answer to the question that you're asking.
Okay. If you don't do that, just to put a bow on this, you're leaving the model up to its own devices to come up with the answer. Right. And, again, if the two people listening take anything away from this, if you're doing that or that's how you think this works, you're doing it wrong. Garbage in, garbage out. Yeah, the expectations for truth and accuracy should be almost zero because that's what you're supplying is zero. Okay, so you build this text document. Move it down the list. Well, and additionally, the other parts you're talking, so by the way, what are we talking about building here? We're talking about building what's called a prompt template, okay?
So that big, huge thing you saw me talk about, like all those slides at the beginning were literally just pieces of a prompt template. So a lot of people think a prompt is just the question or the input. It's actually just one piece of a huge document that is actually getting sent over. Just like if my wife is doing the custom instructions and then her question and those two things get sent over and only one of them is exposed to the user, the other one's behind the scenes. It's almost all of this is behind the scenes. So we've talked about user persona, the input, the question itself, and the answer. There's more things you can send over, actually, which I think is probably makes sense for us to explore that.
And the other things you can send over, as you touched on earlier, were also here's what I don't want you to do when you're answering my question. You know, I don't want you to use flowery language or or, you know, all caps or emojis or whatever it is like this. And those are called constraints. So you'll literally put after it, you know, an all bold constraints colon and then list a bullet list of things you don't want it to do when it's answering. And those are all results of you testing your prompt template like a ton of times before you actually go live. Yeah. And then you get you get results you don't want. You put it in that document. Exactly. OK, that's bingo.
Right. And so now that's that's the testing the things that you don't want it to do. OK. After that is what you just described earlier. So one of my favorite things. This is called few shot. Just think of it as I'm providing examples as well, too. So let me talk about a practical problem that like that can solve. I was doing one of our prompt templates for a customer and I was trying to write these bullet points in a certain format. It was like, you know, it's like a product bullet point like you'd see on like Amazon or something like that. And so it has to be like all caps is the is the first part of it, a colon and then one sentence with a certain tone followed by another sentence.
And then that's it. Right. And so I was trying to go into the prompt instruction, which is other things you can provide is like, Hey, by the way, when you're answering this question, make it all caps and I tried so hard to get it to do that. And it just wouldn't do it with consistency. So then I just said, look, here are five examples of ones that I wrote, or I got from the customer actually in that case that were perfect. And I said, don't like, it's not about the information that's in those examples. That's this few shot learning is what we're doing. It's about the format and the tone. And I specifically say, just pay attention to the format and the tone.
We're not teaching it anything with that other than what the tone or format or style or example might be. Another one is you could literally say, like, you're writing product descriptions. You can go find the greatest product description that a human ever wrote in the history of your customer and say, give me that. And then you literally at the end of your prompt template go, and by the way, this is like a perfect example of what i want you to do and it will be really faithful to that so that's what's called few shot learning those go into your prompt template as well too so in summary just so we don't lose anybody user persona hidden behind the scenes but it's part of your your curl request payload the actual input itself right then uh the answer which is called context that's the ai version of context the actual answer from your data your your database search or your api call or whatever then uh examples or your few shot uh learning examples of what you want the output to either look like or flow like or here's the perfect example not the information that's in it who cares it's it's smart enough to know you're just talking about format or style or something then you send that all over as part of your payload and there's a ton of other stuff you can cram in there but i think that's good and every single one of those goes with every query yes and then also you're
going to get back the queries that you've made before and maybe the answers it's given you before yes so it's a ton yeah so it's going to be a lot of data yeah it is and to to and this is why token windows are so important but you know that that discipline that i just mentioned is so critically important to getting truth and accuracy out of your ai integrations that i'm like i'm so happy to see that like a million token window model is already here in as you indicated before february 2024 that means by like the end of the year like when gbt5 comes out it'll be like a million or something and like when all the rest of them come out they'll all be a million so that basically means like throw it all in there throw everything you can and so the art of prompting is going to be about how much information and instruction can i send to this thing so by the time i tell it who i am and and what what answer or what question i have and what the answers are what i want the answers to look like and maybe some other guardrails and stuff and some other guidance and the constraints and all that, by the time it gets down to completion, you've narrowed that completion probabilistic algorithm to such a small degree that you will for sure get the correct answer that you're looking for.
That I think is the most important foundation of integrating AI into your application. And then when it's all working, you can just say, cure hepatitis and then poof. Well, I mean, sure. Yeah, I mean, maybe. I mean, listen, that will happen at some point. I'm telling you that. It actually, now that it actually is treatable, you can actually get it. If it's not too late, you can actually, there's a drug to actually cure it and it's preventable. It's one of the, it's now a possible disease to eradicate. It's not going to happen in a really long time, but it's a really, really awful disease. I think when you're talking about curing disease, I mean, that's profound, right?
Like that's a profound responsibility to try to lay on something like AI. There's only one. Well, I mean, we've cured smallpox. That's it. That's the only disease in human history to be eradicated. Well, but I think it's, you know, call me a heretic, but here's why I think it's more possible that we cure these things now that we have tools like AI at our disposal. And the reason is because of the, essentially what AI is doing for you is giving you human, like human brain capacity of processing times X or, you know, X, right? Right. So, and we're not talking like twice as smart as a person would be. We're talking like millions of, you know, so like when you think about like as human beings, we can, we have a level of reasoning where we go, all right, well, I'm making a decision and I can at least interpret what the possible consequences might be of the decision.
This could happen or this could happen, right? And that's how we live our lives. And we just do the fork in the road, you know, decisions one after another. I'll throw something at you. There's a book called Thinking Fast and Slow. Did you read that? Oh my God, did I? Not only that, but I'm so glad you brought that up. Oh, my God. It sounds like what you're talking about is that AI can think fast and slow at the same time. Let's, first of all, I can't believe we didn't even do a rehearsal, Matt, and this is so perfect. So here's the deal. That book is, first of all, not about AI, okay? I only learned of it because last year I was doing a bunch of experiments.
You can find him on at iSolutionsAI YouTube, where I was looking at something called this concept of theory of mind. We don't have to get into that now. I suggest we actually do a theory of mind topic in a later episode as a tease. But ultimately, what that led me to was collaborating with a clinical psychologist in the UK who found these experiments. And she was saying, oh, my God, like you're doing clinical psychology experiments against AI. This is so fascinating. Let's talk about it. So her and I did a huge talk about that. it's also on YouTube. And she introduced me to the concept of thinking fast and slow, which is Daniel starts with a K. We'll figure it out. We'll put the link in the YouTube.
Kahneman? Kahneman, yeah. Now I'm sure I knew how to pronounce it. I've read some of his books, so I'm not really sure if he's the author or not. Definitely when you search for, I almost said Google, when you search for the book. So what the premise of that is, is that the brain thinks in two different ways. There's a fast, which is really based on heuristics and like sort of a short-term memory like immediate response but it's genius level right it it connects things together like the example that they give is a is an art curator uh with a ton of experience walking into a museum looking at a statue and says it's a fake where where it would take a huge amount of research and you know all kinds of testing to know but a human can actually sometimes well yes you know connect all these neurons in a crazy way yes so if so the thinking fast is what they refer to as system one uh processing right and then the sinking thinking slow is system two so uh what you just described is um thinking fast with processing power like beyond like you have to be a complete genius like there are humans that have so much experience and so much knowledge and so much context already that they can just
look at something and go, yeah, that's a fake. Every human has that in their little, in their world. We do. No question about it. Now that is our brain. And by the way, that is probably driven more by heuristics than it is about information. It's just like, oh, I've seen this before. I've heard this before. And when I've been in this scenario before, I've seen it. So that's what we're bringing into it. And that, so I think I might've, I think I have some posts that I did about this thinking fast and slow as it pertains to AI and what and the comparison that I was drawing was I think that the chat gbt the zero shot like just ask a question it gives an answer it's all about speed there right it's all about the speed of the response how quickly can we get a response it's not like you ask a question like you did about the hepatitis question earlier and then you walk away from your machine and tomorrow morning you come back and it has the perfect answer like we don't have the patience for that as consumers yeah but today's ai is thinking fast with tons of information well and actually i would say it's using the thinking slow methodology right if i said if i said what's two to the 19th you know you could figure it out by saying two four six eight sixteen thirty two sixty four and you can write it down and you could slowly get an answer right well right but that so yes and no because what you just described uh what our
modern day ai does is pass off to tools that can do that very fast like for example if you ask you integrate in like python to do your math for you or some library that that does that correctly but the true thinking slow as it will be as what my prediction is that there will be our ai experiences will actually be thinking slow which to me means like agency like multiple threads running and research happening and then they come back and compare the models are comparing research with each other and like this actually takes you know minutes or hours or something like that to do it just happens to have so much information and tool set at its disposal that it's thinking fast the system one is actually really impressive because humans we don't have the capacity of that knowledge base available to us yeah exactly we could answer all those but i guess what i'm saying is there's another wave of this coming like sooner than we think that is truly going to be thinking slow that that incorporates multiple models and you never heard of agents like that is still coming and i think is a really important thing and so the book listen to the audiobook read it find a go ask chat gbt for a synopsis because i think it'll really frame up what what things are doing now why it's amazing what we're seeing with ai
and where it's going at the same time and this this was just inspired by a clinical psychologist who understands how the brain works and she drew that she mapped that to me so i it's so great to hear you talk about that i really think that everyone should check that out it's actually one of my predictions of the future i think i did a um that's what i did at the beginning of the year I did a My Predictions in AI for 2024, and I think one of my five was about System 2 processing or System 2 thinking. Right. Okay, sweet. All right, let's get to a real-world example. Were there any other key foundational things that we need to know about? And we're going to go into much more detail in future episodes, but...
Yeah, I think we'll keep today's discussion to the prompt template. How's that? We'll even put that in our description. Like we're talking about prompt templates and, and just in case people have zoned out a little bit with our thinking fast and slow, a prompt template is the stuff that you're sending over as your API payload in Claris, in FileMaker, in your insert from URL, which by the way, is the only tool you need to incorporate AI into your Claris systems. Just want to be clear about that. That's in your curl request, that calculation window that comes in and you're doing the name value pairs and saying this value equals this. you can put as much as you want into that window.
It is only limited by your context window. This is just a review of the things we've talked about so far today. So as long as, and you can test out your prompt template, by the way, when you're writing it and you do a couple of sample searches in your health systems database and it comes back with tons of data, you can go online and there's actually tools that do this, but you can just go online and search for something called the tokenizer and it is a open AI tool. Other people have tools as well. And there's like a Python thing that does it. That just, you can paste all your text into it. It comes back and says, okay, you're at 3,400 tokens right now, right?
FileMaker has a limit of how much data you can have in a crow request, so doesn't it? I would assume that it does. I have not, we have not run into that challenge yet. A couple million? I know, like, you can only have so much text in a calculation, for example. Yeah, it's, and really what it is, is a calculation window, right? So whatever that is. So when, so Clare's platform will have a problem when that limitation is exceeded by the Windows themselves, right? And I know some developers have already hit that in just other things like processing big JSON arrays and stuff like that. Absolutely. And sending data as a parameter over on a caller, like certainly have.
And so that's a very real thing. Now, keep in mind, you can segment your calls. You don't need to send everything over. Like, for example, you could say, hey, write me a product description, and then you take that product description, and then you come back and go, now rewrite it for me as the persona. Like, you could do this in multiple calls, but let's just hope that... You should also be able to make a call to, like, your own AI server, which would have all the information on the other side, right? Yes. Yeah, actually. So in some of the, like, kind of, like, super sophisticated AI implementations that we create, We actually have, we build those as multi-user systems where user management is what's handling the persona.
So we say Chris has an account to access this AI system. Matt has an account. And when Chris logs in, oh, by the way, this is his big persona. And as a secret, also, here's like all the questions Chris has been asking, and we're going to do some modeling and really continue to fine tune and model what he cares about. So that doesn't need to go over with the call that can be applied in a separate call, to your point. Okay. So, so additionally though, to, to, to give like two more examples on the prompt template, you can send guidance as well. Now, the example, you might remember this from my session where I was giving an example of writing product descriptions for one of our customers, right? And their marketing department came back and said, Oh, uh, Hey, cool. You guys are using ai um you know we have guidelines like it's cool that robots are writing these product descriptions and everything but we have guidelines that the human people who are writing these have to follow you know our brand voice there's certain ways that you know there's certain um in different you know uh regions we shape things differently and we respond differently but mostly like hey there's a way we talk about our products and we got to make sure that ai isn't
coming up with its own thing. So in that case, we said, cool, where are those documents? And they sent us over a big document that was called brand voice guideline. Like literally here's the guideline for humans, right? Now I took that PDF. I put it into the aforementioned tokenizer and it told me it was 9,000 tokens. And this was back like a year, a year plus ago when we only had 8K, 8,000 tokens to work with. So I go, well, that's not going to work. So what I did is I took the document and i spent maybe 45 minutes in another ai system and i condensed it down i did this exercise where i said hey ai here's a document parse out all the data from it now and then i want you to go through it and condense it in a lossless manner like that's a keyword that i use so that it doesn't lose any of the information get all the redundancy out of it and get any noise out of there that's not specific instruction and then oh by the way format it in this manner that creates a pattern right yeah i was able to get the 9 000 plus token resource document down to like it was like 3 500 tokens so i said take that 3 500 tokens of information smash that into the prompt template as well and then one last step legal came back because i know everyone's thinking about legal
compliance so i think i've got to address this their legal department which is a big legal department um that said cool it's cool that you guys have robots writing this stuff but we can't make any claims that our products do things that they don't do right and we have these very specific guidelines that human copywriters are supposed to adhere to and so we said guess what matt what did we say give us those guidelines and we did the exact same thing and so in that case our minute our mineral water cures hepatitis yeah hey problem solved uh and half the time you predicted uh but But the point is that instruction, I refer to all that as instruction. If you have the benefit of having that, grab it, go through that condensing process, throw it in your prompt template and you will get amazing, perfect, like guideline specific responses back that you have 100% control over.
So that is our prompt template in review to end the topic and this part of the topic. Who are you? User persona. What question are you asking? um uh what are some of the the guidelines that you need to follow legal ramifications brand or whatever what are examples of the the output formats or tone or style that you want and most importantly what's the answer what's the or what's the data that you want the language model to process to derive your answer and that's what's called context and you send all that over as payload in your api calls and then you get back chef's kiss of responses okay part two uh give me a recent implementation like a small one not a gigantic one that use stuff like this and kind of what they did you can make up a different industry if you don't want to give client stuff away but sure because i think a lot of file maker developers are seeing this as a solution in search of a problem and oh and it's not but no it's not the way to show that it's not is to actually come up with some real world examples of how you made a meaningful difference in an organization? No question. So first off, just to do a really easy one, what I was describing before was several different references to creating a product
description. And I say that because I think what most file market developers or most people that are just starting to explore AI think AI does is just write text, just output text, right? Yeah, or make images. And we can talk about, yeah, we can talk about audio outputs and video outputs and all sorts of cool video inputs and all that kind of stuff later. But we'll just focus on text for today's discussion. And in that case, writing a, as I mentioned, a product description, that was a real like thing for a customer that humans were doing. And then now AI is doing that for them. that is an actual like generative text generative use case example that can be done like a lot of people hear about like writing summaries of meetings that's like table stakes like whatever you can do that with chat gpt it's fine but it's like automatic i mean there's a there's a well zoom has a function for it and then there's i can't remember the other one i use well and that's fathom ai is actually really good oh it's so yeah oh my gosh otter's got it built in zoom's got it built-in like like don't don't be that person that came up with that idea two years late because like you're just gonna like you're it's not a product anymore it's just a feature right yeah so uh but you should at least be doing that like our system at iSolutions all of our meetings are summarized
into who said what what the next steps are and what your tickets need to be created like you should at least be doing that right yeah all right so that's a good example the product the And obviously that's vastly more than you can get by just sitting at a thing and typing prompts into ChatGPT for it, right? I think so, yes. I feel strongly that that's the case. And also, like, yes. So I don't know if we want to blow up that example if it makes sense to people. Like, that is, I think, a really good one. It is, I want something to output into, like, a FileMaker field, you know, show up in a report or something. I want that created for me. Or, you know, if we want to get even more simple, you know, doing daily reports of something like if you're, you know, dispatching people out into the field and they're collecting all this information and then they got to go back to their hotel and they got to like type up these reports or everything.
You don't need to do it that way. You can just say, go do your collection and then just go home at the end of the day. And the system will then take all of your information and write it in the format and derive meaning from it and actionable actionable insights in a manner that might make sense. It's similar to the meeting summary thing, but there's more intelligence and guidance that's necessary to do those kind of things, right? So those, I think, at the level that we're talking about, at least, are probably pretty good examples for what this could do. I mean, and honestly, you know, I'm just trying to hold back on all the crazy cool stuff you can do with AI just to try to keep it simple.
I don't want to get all excited and then knock out of people's minds what they think they're understanding a say here, as we talk about it here, it's not, it's not a intelligence thing. It's just an experience thing. And so, um, you know, uh, product descriptions, bullet points, keywords for SEO, um, you know, uh, other ways to say something, uh, actionable items, uh, insights, insights is my big thing. Like, you know, really what you were describing with the hepatitis data set was, you know, really provide an insight based on the study of this data that would answer my question and to me that is providing not only the data chunks that are coming from your database but giving it access to tools as well too so like you can if for those haven't used python and i was one of those people like two years ago and thanks to the co-pilot i taught myself how to do it i'm an old man and i just submitted oh by the way let me let me just tell all the other old men out there who are afraid to learn new things um in our marketplace which is like every everyone yeah well i know i can say old women because the women in our group are smarter than us old guys and typically younger i've noticed younger as well too i'm talking to the old guys i'm talking to the dudes yeah and what i mean back in the day just for
whatever reason it was just all men when we were yeah look we weren't doing that on purpose i don't you know i had no control over that it's just what the reality of the circumstance was and i'm glad that's not the case anymore but for those who whatever you are who are just um you know stuck in the old paradigm um the i i'm really proud of this i taught myself python using copilot and just in january i submitted you know i'm a sports guy uh sports is this thing where i don't know jocks do it i guess or whatever right but i'm a huge sports guy and frankly i'm glad i am because that's what got me into machine learning and ai in the first place and i submitted to something called the nfl big data bowl where they give you a ton of data and then you have to hypothesize ways to model the data that might be useful to nfl coaches and teams and stuff like that so they just have this open call it's like a hackathon but you have to submit it as python so I used Copilot, taught myself Python and I submitted, dude, I submitted my first, no one helped me besides Copilot.
I tested it. I iterated on it. It took me a couple of months to do it. I'm so proud I did that. Everyone can do that. I'm not a smart dude. Like, I don't know anything about Python. I don't even, I barely, I could do a little bit of JavaScript. I could read it and, you know, carve up other people's JavaScript like crazy, but I don't, I don't know that kind of stuff. And I still was able to figure out enough to understand it. Now, what that led me to understand is even though I can't write it, I was like, oh, my God, Python's got all the tools that I need. ChatGPT can't do math. A Python can do math better than an entire arena full of data scientists, right?
So why don't I have why don't I figure out when I need to send something off to that arena full of data scientists to do some some algorithmic processing? And to your question about hepatitis before, I would probably write some instruction to employ some of those models that I know are science specific train model, you know, uh, Python tools and packages, I would probably incorporate those into my instruction at the same time. So, so thinking it's just a question answer machine and then going, Oh, it got a wrong answer. And then you walk away all satisfied that this is one more thing you don't need to learn is irresponsible because that's not what AI does at all. Yeah. Yeah. I get that.
Well, I know, I know, but I mean, it's, it's definitely worth repeating. Uh, so, so we can, you know, as a file maker community kind of, you know, embrace this, uh, in ways that could make our customers, our databases better and better. Oh, all right. Listen, I, uh, you want to go for another hour or two, or should we, should we wrap it up soon? I think we did a great job, uh, in summary for those that hung in the end here, what we talked about today was prompt templates. the all the stuff that you send over in your api payload in your curl request that you need to send over so that you can get the correct answer and insights including your own data now we can probably talk in upcoming episodes i think we should focus on a couple of things like where do we get that data from obviously file maker databases but we can talk about api calls and i think we should have a conversation about embeddings and vector databases we can have a whole talk conversation about that so important yeah yeah i would love to talk about things like semantic search and then i think in uh upcoming episodes as a tease we'll talk about image inputs super cool video inputs and outputs audio inputs and outputs we're gonna have a lot of fun uh getting into some of the fun weeds i promise that anybody who's leaving this not inspired
will in fact be inspired by upcoming discussions that you and i have i can't wait chris it's always awesome to talk to you. I look forward to our next chat. Thanks, everyone.
Have something to share?
Join me for a conversation on ClarisTalk AI.