1 00:00:02,518 --> 00:00:04,853 Ryan McDonough, how are you this afternoon? 2 00:00:05,250 --> 00:00:07,070 Very good, thank you, how are you Ted? 3 00:00:07,278 --> 00:00:07,749 I'm doing good. 4 00:00:07,749 --> 00:00:10,976 Actually, I guess it's good evening where you are, right? 5 00:00:11,230 --> 00:00:17,723 Indeed, despite the very bright background, it's incredibly dark outside, proper midst of winter here at the moment. 6 00:00:17,794 --> 00:00:18,834 Gotcha. 7 00:00:18,834 --> 00:00:22,956 Well, I have enjoyed your posts on LinkedIn. 8 00:00:22,956 --> 00:00:28,877 You know, I talk a lot about AI because legal innovation is really kind of my focus. 9 00:00:28,877 --> 00:00:32,798 And obviously AI is a big part of that equation. 10 00:00:32,798 --> 00:00:38,720 But, and I think we got a really good agenda for today, but before we jump in, let's get you introduced. 11 00:00:38,760 --> 00:00:42,981 You are currently the head of software engineering at KPMG law. 12 00:00:43,201 --> 00:00:45,382 You also spent some time in legal. 13 00:00:45,972 --> 00:00:49,825 as a software team lead at AG. 14 00:00:50,082 --> 00:00:54,599 Why don't you tell us a little bit about where you're at today and a little bit about your background. 15 00:00:55,005 --> 00:00:56,336 Yeah, sure. 16 00:00:56,336 --> 00:01:05,673 know, prior to sort of working in law, I was working in digital agencies and other small companies, moved into law because there was a really interesting role that I saw, which 17 00:01:05,673 --> 00:01:13,479 was to lead an innovation team, which was a brand new concept that I thought I'd never seen in law, like, you know, trying to make law sexy, which was, which was quite a hard 18 00:01:13,479 --> 00:01:15,901 task, but it's quite an easy one now. 19 00:01:15,941 --> 00:01:18,433 And yeah, so I saw a role like, this, sounds great. 20 00:01:18,433 --> 00:01:20,937 And it was really focusing on how do we... 21 00:01:20,937 --> 00:01:28,830 improve how we deliver to clients and how do we improve how our legal teams work internally and is that building software tools, that just improvement processes? 22 00:01:28,830 --> 00:01:32,541 So it felt like a really good move to move into that position. 23 00:01:32,822 --> 00:01:38,444 And then I moved into KPMG, again focusing on global solutions. 24 00:01:38,444 --> 00:01:48,848 So we're trying to figure out where do we put the most time in put in our priority solutions which have the most effects across all of our member firms across the world. 25 00:01:49,459 --> 00:01:56,875 So again, lot of the time at the moment, as you can imagine, is spent on AI and trying to figure out where to replace this most tactfully in the most useful manner without just 26 00:01:56,875 --> 00:02:00,372 splashing it around like really nearly, I suppose. 27 00:02:01,536 --> 00:02:02,116 Interesting. 28 00:02:02,116 --> 00:02:02,396 Yeah. 29 00:02:02,396 --> 00:02:18,093 KPMG is a frequent topic of conversation here in the States with their recent approval that somehow made it all the way its way to the Supreme court of Arizona to set up shop as 30 00:02:18,093 --> 00:02:19,364 an alternative business structure. 31 00:02:19,364 --> 00:02:25,676 Um, and yeah, I did a presentation at a conference recently. 32 00:02:25,676 --> 00:02:30,092 In fact, it was just three days ago and talked a little bit about 33 00:02:30,092 --> 00:02:38,609 the state of legal innovation and some of the challenges that law firms face truly innovating. 34 00:02:38,629 --> 00:02:52,301 And, you know, I'm a believer that today most of what we're calling innovation in the legal world and the law firm world is really more improvement. 35 00:02:52,321 --> 00:02:58,166 you know, the difference being improvement is incremental and focusing on efficiency and 36 00:02:58,166 --> 00:03:04,139 innovation is really disruptive and kind of rethinking business models and processes. 37 00:03:04,139 --> 00:03:06,351 And not a lot of that has happened in legal. 38 00:03:06,351 --> 00:03:11,654 There is some for sure, but, um, it's been, it's, it's been a lot of improvement. 39 00:03:11,654 --> 00:03:26,424 And, know, I made the case in this presentation that entities like KPMG, which is the smallest of the big four, but still 40 billion in revenue, 4,000 attorneys. 40 00:03:27,061 --> 00:03:42,991 275,000 employees, a global footprint, legions of digital transformation experts and gen AI engineers and Lean Six Sigma black belts and physical assets like the ignition center 41 00:03:42,991 --> 00:03:55,188 in Denver, where you guys bring together clients and consultants from tax audit and advisory and solve problems in a space custom designed for that. 42 00:03:55,192 --> 00:03:57,563 We don't see a lot of that in law firms. 43 00:03:57,563 --> 00:03:59,502 And I did a survey. 44 00:03:59,502 --> 00:04:14,130 I asked after I went through and listed all of those attributes of KPMG and then highlighted that almost two thirds, 63 % of law firm clients either disagree or strongly 45 00:04:14,130 --> 00:04:16,851 disagree that their law firm is innovative. 46 00:04:17,311 --> 00:04:23,874 And even though there's been almost 2000 % growth in the innovation function in the last 10 years, 47 00:04:25,100 --> 00:04:29,652 still almost two thirds of clients don't feel that their law firms are innovative. 48 00:04:29,652 --> 00:04:38,375 And I attribute a lot of that to innovation theater where law firms do things to look innovative, but don't fundamentally commit to change. 49 00:04:39,376 --> 00:04:44,018 so yeah, I, I, think this is a big moment right now. 50 00:04:44,018 --> 00:04:53,562 And I, know, the big four have pushed in, in the U S before with, you know, moderate success, but I think this time is different, different because the means of legal 51 00:04:53,562 --> 00:04:54,582 production, 52 00:04:54,710 --> 00:04:56,233 are really changing with Gen.ai. 53 00:04:56,233 --> 00:04:59,198 So I don't know, what's your take on all that? 54 00:04:59,933 --> 00:05:10,967 Yeah, it's an interesting one because yeah, you say that the innovation theatre and it's definitely something that I sort of see a lot of because you always need to be seen to 55 00:05:10,967 --> 00:05:21,181 continually innovate in an innovation team and it's really hard because there's those tensions between, you know, what is being truly innovative and really changing how the 56 00:05:21,181 --> 00:05:27,103 market works for your law firm and what is doing the bread and butter stuff that needs to be done every day to keep the firm running. 57 00:05:27,103 --> 00:05:28,957 It's how do you decide what to 58 00:05:28,957 --> 00:05:31,948 really invest in when there's no fallback. 59 00:05:31,948 --> 00:05:40,332 And I think that's where KPMG can differ in that sense because like you say, we've got so many teams who do so many incredible things. 60 00:05:40,332 --> 00:05:46,125 It really reminded me of when I moved to a digital agency, were writing more front-end or back-end. 61 00:05:46,125 --> 00:05:48,096 was like, I'm pretty good at both. 62 00:05:48,096 --> 00:05:52,232 And then when I saw what the front-end developers could do there, was like, yeah, I'm probably more back-end. 63 00:05:52,232 --> 00:05:54,783 Like those guys are rock stars at what they do. 64 00:05:54,783 --> 00:06:02,863 And it's the same, like when you think of the ignition center, the UX experts, the AI experts, it's like, I'm really into AI, but there some people are so deep into it. 65 00:06:02,863 --> 00:06:03,583 It's incredible. 66 00:06:03,583 --> 00:06:09,463 And having that expertise available that, you can just go, you know, for this project, we need this. 67 00:06:09,463 --> 00:06:10,543 They'll cross charge. 68 00:06:10,543 --> 00:06:17,423 We get that expertise and that's available to us in a way that, you know, if you're maybe a traditional waterfront, like, we need an expert. 69 00:06:17,423 --> 00:06:17,723 Okay. 70 00:06:17,723 --> 00:06:21,063 Well, let's put a contract out, you know, see if we can get some other three months. 71 00:06:21,063 --> 00:06:23,103 Have you got a business case for this? 72 00:06:23,103 --> 00:06:27,323 you know, who's going to keep an AI staff on staff for a full year. 73 00:06:27,323 --> 00:06:33,823 No one, but actually this KPMG can have that person work on client projects and audit projects and consulting projects. 74 00:06:33,903 --> 00:06:37,183 They have a reason to be there all the time and we can dip into that resource. 75 00:06:37,183 --> 00:06:40,723 I think that's why it's really powerful to have those people on hand. 76 00:06:41,134 --> 00:06:41,574 Yeah. 77 00:06:41,574 --> 00:06:43,374 And you you touched on something. 78 00:06:43,374 --> 00:06:50,914 Uh, so the amount of friction in law firm decision-making is stifling to innovation, right? 79 00:06:50,914 --> 00:06:55,074 Everything is a committee and you've got owners of the business. 80 00:06:55,074 --> 00:07:07,154 And this is true to a certain extent with the big four, the big four are partnership models as well, but they have much less friction and they operate closer to what you would 81 00:07:07,154 --> 00:07:10,414 see in a traditional fortune 500 company. 82 00:07:10,476 --> 00:07:11,736 And I know a little bit about this. 83 00:07:11,736 --> 00:07:17,928 Like I've seen it kind of from the outside, but I spent 10 years at Bank of America and we worked closely with the big four. 84 00:07:17,928 --> 00:07:23,010 was in, um, I was in corporate audit, anti-money laundering, consumer risk. 85 00:07:23,010 --> 00:07:27,621 So we were engaging with the big four all the time in various capacities and. 86 00:07:27,621 --> 00:07:32,672 know, decision-making happens differently there than it does in law firms. 87 00:07:32,672 --> 00:07:39,014 And you guys do have those resources that traditional law firms don't and. 88 00:07:39,416 --> 00:07:41,727 things are going to change so fast. 89 00:07:41,788 --> 00:07:47,292 Innovation is in your culture at KPMG, like true innovation, right? 90 00:07:47,292 --> 00:07:49,453 It's not in Big Law. 91 00:07:49,513 --> 00:07:58,639 And I think they have aspirations of wanting innovation to be integral to their culture, but they're not there today. 92 00:07:58,640 --> 00:08:05,035 So I think the Big Four has some advantages going in. 93 00:08:05,035 --> 00:08:07,680 Big Law has some advantages too, but 94 00:08:07,680 --> 00:08:12,533 It's going to be interesting to see how this tension plays out in the marketplace. 95 00:08:13,149 --> 00:08:14,249 Oh yeah, definitely. 96 00:08:14,289 --> 00:08:22,789 like I said, think it all, a lot of it just comes back to this, you know, having those people available means that, you know, they can do so much other work that, you you just 97 00:08:22,789 --> 00:08:23,609 can't do it all. 98 00:08:23,609 --> 00:08:29,029 You can't go, oh, well we can, you know, get this AI expert to consult with clients and we'll get some billable time off them. 99 00:08:29,029 --> 00:08:32,089 It's just not really feasible having two people doing that. 100 00:08:32,089 --> 00:08:38,669 Whereas we have that capability and we have that interesting building fast and building well. 101 00:08:38,669 --> 00:08:42,769 And we have enough resource to be able to do that at scale, which is incredible. 102 00:08:42,973 --> 00:08:50,733 Again, there are projects underway doing huge AI infrastructure globally, which is needed for a firm like APMG. 103 00:08:50,733 --> 00:08:52,933 And to a far, I think it's needed for most law firms. 104 00:08:52,933 --> 00:09:04,433 If you're of a reasonable size, having an AI architecture in place that would allow business teams to be able to consume it through power automate and power apps in a safe 105 00:09:04,433 --> 00:09:09,273 way that they know is being audited, they know is being have correct guardrails around it. 106 00:09:09,273 --> 00:09:11,709 It's really valuable to have that available. 107 00:09:11,709 --> 00:09:17,609 in a way that means they don't have to spend all the time reinventing the core capability that everyone needs for AI. 108 00:09:17,609 --> 00:09:19,249 We all need auditing. 109 00:09:19,249 --> 00:09:23,129 We all need to log how many tokens are being used for cost effectiveness. 110 00:09:23,969 --> 00:09:28,829 And having that as an AI architecture that just anyone can delve into is so impressive. 111 00:09:28,829 --> 00:09:31,329 It can be a GMG, I can just use this AI capability. 112 00:09:31,329 --> 00:09:35,929 I don't have to spend time doing that core, the lower the water level iceberg work. 113 00:09:35,929 --> 00:09:38,949 I can work on the actual stuff that provides impact. 114 00:09:38,949 --> 00:09:40,167 Whereas I find out a lot of law firms. 115 00:09:40,167 --> 00:09:41,638 they have to reinvent that piece. 116 00:09:41,638 --> 00:09:47,360 They have to go out and find a vendor to offer that piece and you've got to then, you know, spend time and money and effort doing that. 117 00:09:47,360 --> 00:09:50,731 it's just, again, there's no necessary ROI on that. 118 00:09:50,731 --> 00:09:57,143 Whereas I KPMG looks at us, we can see the ROI coming for this and we can absorb this for the next year or two years. 119 00:09:57,143 --> 00:10:00,604 And so we see that ROI arriving for us. 120 00:10:00,846 --> 00:10:01,746 Yeah. 121 00:10:01,746 --> 00:10:11,686 And you know, I think part of the, part of the challenge in the U S too is, and it's not just the U S I think globally in the law firm world is it's very fragmented. 122 00:10:11,686 --> 00:10:16,306 So you have the big four and then you have the AMLAL 200, right? 123 00:10:16,306 --> 00:10:26,946 It's like you've got, so you've got four accounting firms that range from 40 at the low end with KPMG to Deloitte, which is 80 billion. 124 00:10:27,146 --> 00:10:27,648 Um, 125 00:10:27,648 --> 00:10:32,419 And then you've got the biggest law firm, which is a bit of an outlier, which is Kirkland and Ellis. 126 00:10:32,419 --> 00:10:35,910 That's about 7.2, 7.3 billion in revenue. 127 00:10:35,910 --> 00:10:40,732 So KPMG is almost six times the biggest law firm and they're the smallest of the big four. 128 00:10:40,732 --> 00:10:47,604 you know, having those resources is a huge advantage. 129 00:10:47,604 --> 00:10:53,165 And I think an even bigger advantage though, is just the cultural aspects. 130 00:10:53,265 --> 00:10:55,026 It's, it really is hard. 131 00:10:55,026 --> 00:10:56,806 You have to establish. 132 00:10:57,550 --> 00:11:07,496 culture at the top of the organization and at the top of the organization of many law firms are executive committees that are, it's an older demographic typically. 133 00:11:07,496 --> 00:11:08,986 And I've said this a million times. 134 00:11:08,986 --> 00:11:17,501 My listeners are probably tired of hearing it, but a lot of those leaders got there by being the best at lawyering, not because they were necessarily the best leader. 135 00:11:17,721 --> 00:11:18,322 Right. 136 00:11:18,322 --> 00:11:23,284 And I think the decision-making happens differently within the big four. 137 00:11:23,284 --> 00:11:26,794 It's not that you were the best tax auditor. 138 00:11:26,794 --> 00:11:31,762 why you're leading the charge of the tax practice. 139 00:11:31,762 --> 00:11:37,740 But yeah, I think that's another dimension where Big Four has an advantage. 140 00:11:38,172 --> 00:11:41,212 Yeah, think, I think our culture putting mate is really big. 141 00:11:41,212 --> 00:11:51,072 mean, like every Friday, there's, mean, just, in our UK community, there's an AI call that happens every Friday where people are sharing what they've seen in AI and that's across, 142 00:11:51,072 --> 00:11:54,032 you know, consulting, it tags and legal. 143 00:11:54,052 --> 00:11:58,352 We're all chatting about what we've been doing, the ideas we've seen, helping people digest. 144 00:11:58,352 --> 00:12:05,272 Because I always think it's amazing when I see like just how much AI research and like what's the KPMG are putting out. 145 00:12:05,272 --> 00:12:06,392 Who's doing all this? 146 00:12:06,392 --> 00:12:07,752 Like it's crazy. 147 00:12:08,380 --> 00:12:16,360 how big these teams are, that there's just a team dedicated to writing these reports and having like those people just going around sharing stuff like going, oh, cool, I didn't 148 00:12:16,360 --> 00:12:16,880 know that existed. 149 00:12:16,880 --> 00:12:19,260 I can have a look at that and take that. 150 00:12:19,260 --> 00:12:22,120 And there is like a big enough culture to achieve that. 151 00:12:22,120 --> 00:12:24,500 I mean, there's huge networks in KPMG. 152 00:12:24,500 --> 00:12:29,440 it's just like, there's a whole football network and it's just, there's enough people to warrant having that. 153 00:12:29,440 --> 00:12:36,620 And again, other law firms, there's maybe not enough groundswell to build out an AI network or a football network. 154 00:12:36,620 --> 00:12:38,053 So I think it's... 155 00:12:38,053 --> 00:12:48,037 Having that share massive people also helps innovation because you've just got so many ideas bouncing around and so many different viewpoints from different cultures just really 156 00:12:48,037 --> 00:12:50,229 helps spur on more innovation I find. 157 00:12:50,594 --> 00:12:50,864 Yeah. 158 00:12:50,864 --> 00:12:52,575 And you guys have infrastructure as well. 159 00:12:52,575 --> 00:13:06,679 Like I was reading about the KPMD digital gateway that I think started on the tax side of the business, but is now a gateway for some of your legal applications as well. 160 00:13:06,679 --> 00:13:07,840 Is that correct? 161 00:13:07,995 --> 00:13:09,035 That's correct, yeah. 162 00:13:09,035 --> 00:13:11,995 So it started off as a digital gateway for tax. 163 00:13:11,995 --> 00:13:14,295 We've just got digital gateway at that point. 164 00:13:14,295 --> 00:13:16,935 But now we have the digital gateway for law. 165 00:13:16,935 --> 00:13:23,175 So it's basically it's like a personalized entry point for accessing all the KPMG technology and services. 166 00:13:23,235 --> 00:13:29,595 So, you know, if you're consuming hopefully many different services from KPMG, they're all in one place. 167 00:13:29,595 --> 00:13:34,695 Again, with different levels of integration and performance that's included into them. 168 00:13:34,695 --> 00:13:36,579 But you know, like we've built out a... 169 00:13:36,579 --> 00:13:39,300 international business reorganization platform. 170 00:13:39,541 --> 00:13:41,362 Sometimes hard to say that quickly. 171 00:13:41,362 --> 00:13:45,705 And we can service information from that inside the digital gateway for more. 172 00:13:45,705 --> 00:13:48,907 So it means that, you know, the client can go onto digital gateway. 173 00:13:48,907 --> 00:13:52,179 They can see all the latest reports that KPMG have put out. 174 00:13:52,179 --> 00:13:55,592 They can see the horizon scanning and go, actually, how's my program? 175 00:13:55,592 --> 00:13:56,953 How's my project progressing? 176 00:13:56,953 --> 00:13:57,473 Click on that. 177 00:13:57,473 --> 00:13:59,394 I can see what's progressing there. 178 00:13:59,474 --> 00:14:00,175 And then I'm done. 179 00:14:00,175 --> 00:14:01,726 I don't have to go off to a different platform. 180 00:14:01,726 --> 00:14:03,837 I don't have to do 58 clicks to get there. 181 00:14:03,837 --> 00:14:04,928 It's all in one place. 182 00:14:04,928 --> 00:14:05,658 So 183 00:14:05,679 --> 00:14:09,761 we're really trying to focus it on that sort of like personalized entry point piece. 184 00:14:10,161 --> 00:14:14,814 And again, it's a really good place because again, there's so many developers working on digital gateway. 185 00:14:14,814 --> 00:14:22,348 So again, when we're looking at ideas of how can we do this and how can we expand, know, there's so many people working on Gen.AI for digital gateway. 186 00:14:22,348 --> 00:14:27,731 It's a really big focus they've got at the moment allowing the clients and our internal teams to use Gen.AI. 187 00:14:27,731 --> 00:14:34,464 So we're trying to bring all those capabilities that clients are talking about and asking for and giving it to them as well. 188 00:14:35,150 --> 00:14:35,640 It's interesting. 189 00:14:35,640 --> 00:14:41,355 So we at info dash started our journey on the internet side. 190 00:14:41,355 --> 00:14:43,467 So we're pretty well established there. 191 00:14:43,467 --> 00:14:49,561 Been doing the work as consultants for 14 years before we productized three years ago. 192 00:14:49,562 --> 00:14:51,443 So we've got, and we've got a great install base. 193 00:14:51,443 --> 00:15:05,314 We just launched an extranet platform, which sounds similar to your digital gateway and the benefit of our approach, because we also see scenarios where 194 00:15:05,599 --> 00:15:08,691 Law firm clients are going to want access. 195 00:15:08,691 --> 00:15:22,161 like the, the incumbent in vendor in the space is high Q high Q is a very ring fenced standalone hosted solution that operates completely outside of the law firms. 196 00:15:22,161 --> 00:15:23,782 Technology environment. 197 00:15:23,782 --> 00:15:25,884 We think there's a better way. 198 00:15:25,884 --> 00:15:34,286 So we have built info dash extra net that gets deployed in the law firms tenant and has access to. 199 00:15:34,286 --> 00:15:37,026 built in integrations with all the backend systems. 200 00:15:37,266 --> 00:15:48,446 So finance, you know, analytics, BI practice management, document management, experience management, CRM, HR, IS so we can pull together. 201 00:15:48,446 --> 00:15:57,586 So if you're a law firm client, you can log in, uh, to one of your matter sites and you can see the entire legal team that is billed time to your matter. 202 00:15:57,586 --> 00:16:02,306 You can see their experience, who they clerk for, what bars they're admitted to. 203 00:16:02,306 --> 00:16:03,362 You can see 204 00:16:03,362 --> 00:16:11,548 details about your engagement letters, any documents related to the matter, any dockets associated with the matter. 205 00:16:12,569 --> 00:16:24,578 So we are betting big on law firms wanting to compete with things like Digital Gateway so they can provide access because today it just doesn't happen. 206 00:16:24,578 --> 00:16:29,581 It's really hard to get information in and out of HiQ on purpose, right? 207 00:16:29,750 --> 00:16:34,002 There's a lot of, yeah, there's definitely a lot of data silos with a lot of legal tech tools. 208 00:16:34,002 --> 00:16:41,516 And I think that's just with the intention of retain clients because you simply can't get data in and out of them. 209 00:16:41,516 --> 00:16:50,661 And I think that the idea of having connections to all the systems in one place, because I think, I'm saying like, you'll be going full AI for everything, but it's having that 210 00:16:50,661 --> 00:16:55,724 going, AI would have the context of the whole issue for the client. 211 00:16:56,064 --> 00:16:57,675 What does the lawyer know about the client? 212 00:16:57,675 --> 00:16:59,536 What do we know from our horizon scanning? 213 00:16:59,536 --> 00:17:01,726 What do we know from all the other databases we have? 214 00:17:01,726 --> 00:17:04,508 What people have been talking about the client recently? 215 00:17:04,508 --> 00:17:06,358 And then that's all accessible at that point. 216 00:17:06,358 --> 00:17:15,604 So you have that, you know, that's canonical source of truth of the client and all their requirements rather than again, saying, well, some of this is in this system and some of 217 00:17:15,604 --> 00:17:16,753 this is in this system. 218 00:17:16,753 --> 00:17:21,215 actually we're going to have to do a huge uplift to integrate them all into like some sort of data wake. 219 00:17:21,215 --> 00:17:24,476 It makes sense to go down the approach that you are. 220 00:17:24,578 --> 00:17:26,039 Yeah, we think so. 221 00:17:26,039 --> 00:17:35,816 And we think over time, the competitive pressures in the landscape are really going to force both the vendors, the legal tech vendors and law firms to kind of reevaluate the 222 00:17:35,816 --> 00:17:39,688 current strategy, which is clients don't really get what they need today. 223 00:17:39,989 --> 00:17:47,334 Very few, very few clients even share financial data with their, with their customers through, through extra nets. 224 00:17:47,334 --> 00:17:49,765 Like maybe they have the last X number of bills. 225 00:17:49,765 --> 00:17:52,037 There's very little work in process. 226 00:17:52,037 --> 00:17:54,190 That data has to get massaged. 227 00:17:54,190 --> 00:18:04,013 Um, you know, time entry data before it gets, it gets released, uh, to, clients, but clients want transparency and they don't have it today. 228 00:18:04,013 --> 00:18:14,556 So yeah, we're putting our chips on the table saying, you know what, you've been able to get away with this because you just have big, big law has been very resilient over the 229 00:18:14,556 --> 00:18:15,486 last 50 years. 230 00:18:15,486 --> 00:18:15,906 Right. 231 00:18:15,906 --> 00:18:19,057 But the, think times are a changing. 232 00:18:19,057 --> 00:18:23,678 Um, I think there's going to be new competitive pressures that are going to 233 00:18:23,886 --> 00:18:26,259 cause some different ways of thinking. 234 00:18:27,064 --> 00:18:35,664 I think there's been that sort of push by what people's expectations are around the applications and even the billing piece. 235 00:18:35,664 --> 00:18:47,324 again, I remember when my first job was at a law firm many years ago and a partner turned up and went, turn this into a website, gave me a form and I need a website and I didn't 236 00:18:47,324 --> 00:18:51,004 see them for six months and they turned back up and they were like, wow, that's amazing. 237 00:18:51,004 --> 00:18:53,879 People are amazed that a website could exist back then. 238 00:18:53,879 --> 00:18:58,659 But, you know, people are so used to like the applications on the phones and the quality that they have. 239 00:18:58,659 --> 00:19:04,059 And again, being able to see the bills, they go, well, I can see my bill for Google suite and I can see my bill for Uber. 240 00:19:04,059 --> 00:19:06,159 So why can't I see my legal bills? 241 00:19:06,279 --> 00:19:13,479 And it's, and again, it's that, it's that thing of going, well, you know, if I'm requesting a bill off the legal team, that's time out of the legal team's day. 242 00:19:13,479 --> 00:19:16,439 You know, just responding to that particular request. 243 00:19:16,439 --> 00:19:18,039 I'm going, oh, where's my bill? 244 00:19:18,039 --> 00:19:19,819 Can you forward it on to me? 245 00:19:20,310 --> 00:19:27,916 non-billable time and it's you know, useless and it's taking you out of your contact switching, especially if it's like a really big client and they're, know, really important 246 00:19:27,916 --> 00:19:34,791 and you're going, oh, I have to that email, have to, you know, stop doing what I'm doing, get this bill from whatever system and send it off to them. 247 00:19:35,012 --> 00:19:43,168 Again, maybe if that person goes on holiday and then someone else wants that same request, it's putting the data in front of the client so they can self-serve as much as is possible 248 00:19:43,168 --> 00:19:49,323 because clients, I think, are getting more and more used to that concept now and I think they prefer it from lot of issues. 249 00:19:49,474 --> 00:19:51,495 Yeah, that totally makes sense. 250 00:19:51,495 --> 00:20:05,597 Well, you and I were going to talk a little bit about AI today and specifically we were going to talk about benchmarks and you know, I have conflicting feelings about the 251 00:20:05,597 --> 00:20:06,479 benchmarks. 252 00:20:06,479 --> 00:20:19,362 I feel like in this rush for the first AI vendor to plant their flag on Planet AGI that the benchmarks have really focused 253 00:20:19,362 --> 00:20:30,998 That's been a focal point and we see, you know, it can do all these PhD level tasks, but it still can't summarize my spreadsheet, my pivot table, you know, just the basics of, you 254 00:20:30,998 --> 00:20:36,050 know, blocking and tackling use cases are still very hit or miss. 255 00:20:36,210 --> 00:20:45,014 So like, what are you, what is your take on like the limitations of the current benchmarks, you know, for like real world legal applications? 256 00:20:45,207 --> 00:20:52,487 Yeah, it's interesting because like we say, I we've seen the VALS AI piece recently and that's a big step improvement on how it is. 257 00:20:52,487 --> 00:20:58,367 But I mean, a lot of the academic AI benchmarks just aren't designed for real world legal work. 258 00:20:58,367 --> 00:21:07,847 They're testing sort of AI on very isolated tasks like extracting clauses, summarizing documents and answering simple legal questions that maybe I could even have a go at. 259 00:21:07,847 --> 00:21:11,007 know, law isn't these disconnected tasks. 260 00:21:11,007 --> 00:21:12,502 It's about context. 261 00:21:12,502 --> 00:21:17,846 and reasoning and linking those different bits of information across all these different documents. 262 00:21:18,067 --> 00:21:22,350 it's, know, mean, like, you know, a model might be able to extract a termination clause. 263 00:21:22,402 --> 00:21:28,886 But in practice, like, the lawyers want to know, like, how does this interact with liability caps and notice periods and local regulations? 264 00:21:28,886 --> 00:21:32,970 And are there similar clauses that contradict it in the same document? 265 00:21:32,970 --> 00:21:36,083 And does it actually align with your client's risk profile? 266 00:21:36,083 --> 00:21:40,394 And that's where the academic benchmarks fall down because they're not measuring. 267 00:21:40,394 --> 00:21:47,299 that level of legal reasoning, document complexity and just the real world constraints that we face every day with documents. 268 00:21:47,299 --> 00:21:52,988 I mean, even when I was building AI systems very early on, when ChatGPT came out, the API was available. 269 00:21:52,988 --> 00:21:57,496 I was like, cool, I'll get this Word document that someone's made for me and I'll analyze that. 270 00:21:57,496 --> 00:22:04,251 And it was great because it was a Word document and it was exactly structured how I needed it to be, but that's not really reflective of the real world. 271 00:22:04,251 --> 00:22:08,954 It's not a handwritten annotations on a document and drawings and... 272 00:22:09,000 --> 00:22:14,392 literal red lines on the document from versions of it with signatures overlapping words. 273 00:22:14,392 --> 00:22:22,864 it's, know, as a human I go, well, that probably says director because I can see the word D-I-R and then O-R at the end of it, but part of the signature is going over it. 274 00:22:22,864 --> 00:22:25,615 But AI was like, don't know what that is. 275 00:22:25,615 --> 00:22:27,996 Very illiterate, I have no idea what that is. 276 00:22:27,996 --> 00:22:33,798 again, need, firms need to basically have their own benchmarks, I feel, to reflect their own workflows. 277 00:22:34,296 --> 00:22:37,987 that's how the conversation actually started. 278 00:22:38,927 --> 00:22:41,648 There is a legal benchmark out there. 279 00:22:41,648 --> 00:22:43,929 There's an associated GitHub repository. 280 00:22:43,929 --> 00:22:45,949 I saw the contributors. 281 00:22:45,949 --> 00:22:51,811 think of which, were you one or maybe you just pointed a link to that GitHub repository? 282 00:22:53,091 --> 00:22:53,572 Okay. 283 00:22:53,572 --> 00:23:02,554 Well, talk to us a little bit about developing custom benchmarks that better reflect legal use cases. 284 00:23:02,888 --> 00:23:03,618 Yeah, sure. 285 00:23:03,618 --> 00:23:11,863 So I feel like, you know, when you're trying to sort of move just beyond vendor demos, you know, where they're very highly polished and I've done this myself demoing tools 286 00:23:11,863 --> 00:23:15,744 internally, I have a very strict script I wanted to keep too. 287 00:23:16,225 --> 00:23:20,227 But when you want to try and build your own framework, you sort of base it on the real tasks. 288 00:23:20,227 --> 00:23:25,290 So sitting down with the team and figuring out what those are, it's like, you know, testing it on actual legal documents. 289 00:23:25,290 --> 00:23:31,669 So not just taking the ones that vendors are supplying you, it's finding the ones that are really reflective and go into the team and you know. 290 00:23:31,669 --> 00:23:34,109 What's been a weird document you've seen recently? 291 00:23:34,109 --> 00:23:40,989 Like I was trying to analysis backwards as an AG of a document, trying to do machine learning analysis of a document structure. 292 00:23:40,989 --> 00:23:50,429 And it was great and all that precedence that we had, but then I got a client document and the only way they differentiated between the section headers and the clauses was they used 293 00:23:50,429 --> 00:23:51,769 purple text. 294 00:23:51,809 --> 00:23:53,489 And that was the only differentiation. 295 00:23:53,489 --> 00:23:55,209 wasn't like one dot whatever. 296 00:23:55,209 --> 00:23:56,809 It was just a bit of purple text. 297 00:23:56,809 --> 00:23:58,429 And you're like, well, how do I train for that? 298 00:23:58,429 --> 00:23:59,709 That is crazy. 299 00:23:59,765 --> 00:24:07,205 But those are the kind of documents you want to use because you want to see if AI can actually handle the clients insane documents that have been handed over or just documents 300 00:24:07,205 --> 00:24:11,285 that you've got from 15 years ago that are actually materially relevant. 301 00:24:11,685 --> 00:24:14,945 And then you have to measure that impact. 302 00:24:14,945 --> 00:24:21,345 like, it reducing the time spent on review or is it just adding another layer of verification on top of it at the moment? 303 00:24:21,345 --> 00:24:26,684 And if it is just adding another layer of verification on top of it, again, making a note of that because again, 304 00:24:26,684 --> 00:24:27,915 That may change over time. 305 00:24:27,915 --> 00:24:35,814 may need to revisit these tools and the benchmarks that you have and go, well, actually this was previously something we had to verify, but now we don't have to verify it. 306 00:24:35,814 --> 00:24:37,453 So actually this is more applicable. 307 00:24:37,453 --> 00:24:42,468 And then again, evaluating the accuracy is so important in the real world conditions. 308 00:24:42,468 --> 00:24:48,443 It's, know, scan PDFs, handwritten notes, multiple jurisdictions, redacted clauses. 309 00:24:48,443 --> 00:24:52,226 How is it handling those within its system? 310 00:24:52,226 --> 00:24:53,998 And those are the things we really need to evaluate. 311 00:24:53,998 --> 00:24:54,808 And then... 312 00:24:55,465 --> 00:25:03,131 I think as well comparing the vendor AI to a well-promoted LLM, you know, and checking if the product actually adds value. 313 00:25:03,131 --> 00:25:12,558 Because again, these strong benchmarks should really assess like the accuracy, the consistency and the speed and the ability to handle these messy complex documents beyond 314 00:25:12,558 --> 00:25:15,481 just saying, can you get a simple question right? 315 00:25:15,481 --> 00:25:18,993 asking who are the parties, you know, what is the duration of the contracts? 316 00:25:18,993 --> 00:25:22,666 Like we can all do that, know, chat GPT can do that incredibly well. 317 00:25:22,666 --> 00:25:24,788 How well can you actually handle the real world legal? 318 00:25:24,788 --> 00:25:25,783 questions. 319 00:25:26,092 --> 00:25:26,642 Yeah. 320 00:25:26,642 --> 00:25:37,289 What about like identifying low risk, high value use cases where AI can like immediately benefit legal teams? 321 00:25:37,289 --> 00:25:40,020 Like what's a good, what's a good strategy for that? 322 00:25:40,756 --> 00:25:41,476 Yeah. 323 00:25:41,476 --> 00:25:51,316 So I think, mean, well, the easiest immediate high value use cases that I personally see is trying to automate the non-millipede stuff, like the time consuming stuff and, you 324 00:25:51,316 --> 00:25:58,136 know, the stuff that, you know, when people were watching suits back in the day, they weren't looking at going, oh, I wish I was going to do that task one day. 325 00:25:58,136 --> 00:26:05,216 I hope I could be buried in paperwork or hope I'm just sat there triaging requests that are coming in like, no, they were going, I want to do the call cases. 326 00:26:05,216 --> 00:26:06,756 I want to do all the fun stuff. 327 00:26:06,756 --> 00:26:10,656 And so if, you know, if something as simple as like triaging incoming emails. 328 00:26:10,684 --> 00:26:14,566 shift that unbillable time into more productive revenue generating work. 329 00:26:14,566 --> 00:26:25,693 again, if you're receiving hundreds of clients or emails daily, some urgent, some routine, some completely irrelevant, AI can categorize and prioritize those on urgency, on the 330 00:26:25,693 --> 00:26:27,274 client, on the subject matter. 331 00:26:27,274 --> 00:26:35,368 And again, if you've got a system like you were talking about before where you have the finance context, you can actually prioritize on how much are we billing this client? 332 00:26:35,368 --> 00:26:37,073 How often are we billing this client? 333 00:26:37,073 --> 00:26:38,994 and do urgency based on that as well. 334 00:26:38,994 --> 00:26:47,801 You know, can extract the key deadlines and action items from the email threads and then the intelligent routing, which is what I really love about it is going, well, who's the 335 00:26:47,801 --> 00:26:52,134 right lawyer or who's the right department to be doing this based on the content that's coming in? 336 00:26:52,134 --> 00:26:54,556 And again, like summarizing length of the email chains. 337 00:26:54,556 --> 00:26:56,383 Like I was using that recently. 338 00:26:56,383 --> 00:27:04,146 I just got off an extra couple of weeks of facility leave and I was taking these big email chains and getting AI to summarize, you know, what was the email chain? 339 00:27:04,146 --> 00:27:08,577 Am I still needed in this email chain, drafting an email to confirm if I'm not needed or not? 340 00:27:08,577 --> 00:27:16,519 And it was amazing to think of like, compared to what I was doing a year ago, where I was just working up the chain from the bottom of the email, getting to the second to last 341 00:27:16,519 --> 00:27:21,160 email and realizing I'm not needed anymore and wasting 15 minutes of my day. 342 00:27:21,160 --> 00:27:29,732 And obviously like, you know, these sort of seem like small efficiency gains, but you know, shaping minutes off every email review, compounds over and over, much like your 343 00:27:29,732 --> 00:27:33,283 pension does, into hundreds of hours saved each week. 344 00:27:33,315 --> 00:27:42,860 And if that can be redirected to billable work and high strategic value tasks, then it's a really good way to be doing it because like I've always said, AI should be replacing 345 00:27:42,860 --> 00:27:48,603 Claudius, but hopefully it should quite quickly eliminate those repetitive administrative tasks. 346 00:27:48,603 --> 00:27:52,085 allowing the legal professionals to work on what actually moves the needle for the clients. 347 00:27:52,085 --> 00:27:54,977 know, the clients are going, that's really good email tree. 348 00:27:54,977 --> 00:27:56,507 I shouldn't have done that. 349 00:27:56,648 --> 00:27:59,429 No one's saying that, but actually going, that was great. 350 00:27:59,429 --> 00:28:00,522 doctor really drafted it for me. 351 00:28:00,522 --> 00:28:01,142 That was really quick. 352 00:28:01,142 --> 00:28:03,079 How did you manage to spend so much time on that? 353 00:28:03,079 --> 00:28:05,739 Well, I wasn't triaging emails all day. 354 00:28:06,594 --> 00:28:07,775 Yeah, good, good point. 355 00:28:07,775 --> 00:28:21,402 I've been, I've been beating that drum for a while in terms of leveraging AI as a starting point, leveraging AI for business of law use cases is a good risk reward balance, right? 356 00:28:21,402 --> 00:28:25,104 Because it allows, it's a, it's an incremental step forward. 357 00:28:25,185 --> 00:28:35,050 It's a, there's still an opportunity cost that you're eliminating in terms of administrative time and you don't have to 358 00:28:35,084 --> 00:28:46,302 worry about client data in most scenarios and all of the OCGs that I saw a really interesting survey from Legal Value Network. 359 00:28:46,302 --> 00:28:51,796 do a legal project management survey. 360 00:28:51,796 --> 00:29:02,798 And one of the questions was, what percentage of your clients prohibit or discourage the use of GEN.ai on the 361 00:29:02,798 --> 00:29:03,998 resolution of your matters. 362 00:29:03,998 --> 00:29:17,458 And the answer was 42 % either prohibit or discourage the use of AI, which I thought was shockingly high because I've seen other numbers where clients, in fact, I think it was, 363 00:29:17,458 --> 00:29:22,238 um, it was from the, uh, blickstein groups. 364 00:29:22,238 --> 00:29:31,578 do a law department organization, LDO survey, uh, every year and almost 60 % of clients said, 365 00:29:32,110 --> 00:29:38,710 that law firms don't use technology enough to reduce costs. 366 00:29:38,710 --> 00:29:42,450 And it's like, all right, those are two conflicting messages. 367 00:29:42,450 --> 00:29:52,210 If you've got clients who want you to use tech to eliminate the costs, but then are discouraging or prohibiting you from using the tech to reduce the cost, we have a little 368 00:29:52,210 --> 00:29:55,610 bit of a tension there, which I don't really know what to make. 369 00:29:55,610 --> 00:29:55,890 I don't know. 370 00:29:55,890 --> 00:29:56,978 You got any insight on that? 371 00:29:56,978 --> 00:30:07,058 I mean, I'm very much getting flashbacks of when there was a little chores around the client turns around the The morphers cloud is like, well, I don't want to use the cloud. 372 00:30:07,098 --> 00:30:08,338 What about my security? 373 00:30:08,338 --> 00:30:15,338 I was like, I'm not sure where you think your emails go between you and us, but it's going to be the cloud at some point. 374 00:30:15,338 --> 00:30:20,838 So, and again, I think it's positioning, I it's positioning the tech in the correct format. 375 00:30:20,838 --> 00:30:22,963 Cause I think there was so much like foot. 376 00:30:22,963 --> 00:30:27,345 like the fear, uncertainty and doubt around the cloud and around AI. 377 00:30:27,345 --> 00:30:36,228 And it's trying to position itself the cloud going, know, we're not sitting here and going, the AI is doing this entire job unassisted and no one's looking at it. 378 00:30:36,228 --> 00:30:46,472 What we're trying to do here is we're trying to reduce the pain points that we're all facing in law, but currently he's taking up our time doing your work, getting us to do the 379 00:30:46,472 --> 00:30:49,093 interesting stuff that you actually value here. 380 00:30:49,113 --> 00:30:52,854 I was chatting to a friend who works at Adelshaw's and they were talking. 381 00:30:52,854 --> 00:31:02,357 about projects they've done in the past and something that's similar to WeHoTel, which is around, you know, when you look at a huge data room, you can use generative AI to 382 00:31:02,357 --> 00:31:08,818 basically take all the documents, look at them, summarize them, check is the title actually correct? 383 00:31:08,878 --> 00:31:15,020 You know, like, is this actually an NDA or is this actually, you know, an employment contract? 384 00:31:15,100 --> 00:31:22,442 And reviewing the contents of the document and going actually, let's probably update this title, like from NDA to employment contract. 385 00:31:22,961 --> 00:31:26,301 And like, that's a huge task for someone to be doing. 386 00:31:26,301 --> 00:31:33,021 You know, like you give up to power legal and that's probably like three weeks worth of work to get through like 10,000 documents or so. 387 00:31:33,541 --> 00:31:39,101 But using AI to do that seems like a really easy, again, low hanging fruit piece to try and do. 388 00:31:39,101 --> 00:31:41,541 It's high impact, low risk. 389 00:31:41,781 --> 00:31:44,521 And you can also do document summarization at the same time. 390 00:31:44,521 --> 00:31:48,881 So can go, here's your document, we've fixed the title, we've fixed the categorization of it. 391 00:31:48,881 --> 00:31:50,381 Here's a summary of the document. 392 00:31:50,381 --> 00:31:50,881 Cool. 393 00:31:50,881 --> 00:31:52,041 Off we go. 394 00:31:52,221 --> 00:32:00,348 I think it's all about the framing of AI because I think AI landed in a way that no other legal tech ever has. 395 00:32:00,969 --> 00:32:03,471 Document automation was never on the BBC News. 396 00:32:03,471 --> 00:32:10,316 There was never an emergency worldwide conference hosted by the primaries of the UK around document automation. 397 00:32:10,537 --> 00:32:13,279 People had a lot more visibility into it. 398 00:32:13,279 --> 00:32:15,921 And again, it's on their devices so quickly. 399 00:32:15,921 --> 00:32:20,685 Alexa are doing their Alexa Plus in the coming months or so in the US where... 400 00:32:20,785 --> 00:32:24,865 Genitive AI will be in everyone's homes all of sudden. 401 00:32:25,665 --> 00:32:32,725 And I think that just then gives people expectations, but also gives them that issue where they go, well, hold on, what is this actually doing? 402 00:32:32,725 --> 00:32:36,905 Because again, it is still a black box for a lot of internal testing purposes. 403 00:32:37,545 --> 00:32:44,665 then researchers are really trying to delve into, how does it make the decisions that it makes and why does it anchor onto certain concepts that it does? 404 00:32:44,665 --> 00:32:48,081 But it's quite hard to do that still, so it is. 405 00:32:48,081 --> 00:32:53,341 In the same way that it's quite hard sometimes for a human to explain what they're doing and how they've approached a particular task. 406 00:32:53,541 --> 00:33:00,821 So I think that's main thing is how we frame AI and how we frame where we're using AI to really get people comfortable with it. 407 00:33:00,821 --> 00:33:08,141 Because like you say, if we're not using like, you know, we've exhausted a lot of the tech that was bringing benefits to people. 408 00:33:08,141 --> 00:33:12,541 And you know, we have teams, have emails, we're not doing postal stuff anymore. 409 00:33:12,541 --> 00:33:13,096 So. 410 00:33:13,296 --> 00:33:16,016 AI is the next big revolutionary for me. 411 00:33:16,016 --> 00:33:16,996 It's huge. 412 00:33:16,996 --> 00:33:21,196 It feels the same way that the internet did back when I was a kid. 413 00:33:21,196 --> 00:33:22,316 was really excited. 414 00:33:22,316 --> 00:33:28,976 When I was a kid, the internet came, was like, oh my goodness, I can read, can talk to people, I can do this, I can do that, I can go on forums. 415 00:33:29,036 --> 00:33:31,996 And AI sort of had that same sort of excitement for me. 416 00:33:31,996 --> 00:33:33,856 Like, this is incredible. 417 00:33:34,336 --> 00:33:37,256 It's able to do things that we've not seen before. 418 00:33:37,576 --> 00:33:42,864 mean, talking beyond sort of tech, the vision capabilities, you could basically allow. 419 00:33:42,864 --> 00:33:46,204 blind people to read newspapers and give them context. 420 00:33:46,524 --> 00:33:49,144 Even of the stuff which is very visually heavy. 421 00:33:49,144 --> 00:33:56,284 I remember I was at my parents' house and there was a wine review and there was a table with some wines, some prizes, some texts underneath them. 422 00:33:56,284 --> 00:34:01,844 And I took a picture and asked ChatGVT to structure this in a way that I could then read back out. 423 00:34:01,844 --> 00:34:07,684 And it took these sort of inferences, like there was red text inferred ahead of it, but it wasn't like a structured table. 424 00:34:07,684 --> 00:34:11,384 It was really sort of quite an informal way of presenting the data. 425 00:34:11,736 --> 00:34:19,146 And that's an amazing accessibility piece that generative AI has enabled that wasn't available before. 426 00:34:19,146 --> 00:34:26,435 again, I think it's sort of showing those real benefits to people and showing where we can start to use it tactfully in our workflows. 427 00:34:26,988 --> 00:34:33,861 Yeah, there's all sorts of interesting insights that previously would never be discovered. 428 00:34:34,441 --> 00:34:42,215 I went to TLTF this year in Miami and in the lobby of the Ritz Carlton, they had a gingerbread house, you know, it was December. 429 00:34:42,215 --> 00:34:46,327 They had a gingerbread house, the size of like a Volkswagen Beetle. 430 00:34:46,327 --> 00:34:48,568 And my wife was like, that's so cool. 431 00:34:48,568 --> 00:34:53,930 You know, and they had the list of ingredients that was required to make it. 432 00:34:54,178 --> 00:34:55,699 So I was like, you know what'd be interesting? 433 00:34:55,699 --> 00:34:59,441 I was like, I wonder how many people that this house could feed. 434 00:34:59,501 --> 00:35:08,876 And so I took a picture of the ingredient list and asked chat GPT, how many cookies could you make with the same number of ingredients? 435 00:35:08,876 --> 00:35:10,047 And the answer was like 26,000. 436 00:35:10,047 --> 00:35:13,988 Like there were 26,000 cookies. 437 00:35:14,089 --> 00:35:15,820 It completely changed my wife's perspective. 438 00:35:15,820 --> 00:35:19,252 She's like, she went from, that's amazing to that's horrible. 439 00:35:19,252 --> 00:35:20,142 What a waste. 440 00:35:20,142 --> 00:35:23,842 Like we got people starving in the streets and we have this 441 00:35:23,842 --> 00:35:26,624 food here that we're using as decoration. 442 00:35:26,825 --> 00:35:38,644 And you know, I've, uh, I was talking to another guy down in Miami, the same conference, and he was, uh, his basement flooded and he was retired. 443 00:35:38,644 --> 00:35:50,654 had to retile his basement and he had all these boxes of tiles and he took a picture and asked chat GPT how much in, in, you know, square footage. 444 00:35:50,654 --> 00:35:53,846 And he also took a picture of the floor plan. 445 00:35:54,070 --> 00:36:00,292 And you know, how much, surplus or, you know, short was he with the tie and it gave him the answer. 446 00:36:00,292 --> 00:36:10,645 And it's just like, you know, again, that's the, those aren't business scenarios, but just w but just by taking a picture, can get, and it re you know, it read the box and the size, 447 00:36:10,645 --> 00:36:13,055 the dimensions of the tiles and the number of tiles in the boxes. 448 00:36:13,055 --> 00:36:15,556 And he had like 50 boxes. 449 00:36:15,676 --> 00:36:17,747 So it's, uh, it's incredible. 450 00:36:17,747 --> 00:36:22,978 The things that you can gain insight on that previously would have been out of reach. 451 00:36:23,630 --> 00:36:32,350 Yeah, and I think it's even when you go back to the business world, when I was looking at the Alexa memo recently and they saying like, the plan is for them to be a bit more 452 00:36:32,350 --> 00:36:33,850 proactive going forward. 453 00:36:33,850 --> 00:36:37,550 So you could be like, well, have a look at my calendar and see what's coming up. 454 00:36:37,750 --> 00:36:44,310 you know, you know, I've been having conversations, have I said, oh, let's catch up when I'm in London next, you know, so when am I in London next? 455 00:36:44,310 --> 00:36:49,935 Can we check their calendar as well and do all those kinds of things, which takes a lot of work. 456 00:36:49,935 --> 00:36:56,195 It tends to be why people don't end up seeing each other as much because I forgot to do that extra stuff, I didn't have the time. 457 00:36:56,195 --> 00:37:03,995 Well, having that sort of AI system being more proactive in a privacy preserving way just seems like such a cool thing to have. 458 00:37:03,995 --> 00:37:06,795 It could be like, know, do I need to buy any birthday presents? 459 00:37:07,215 --> 00:37:11,755 well, I can see that you're going to be going to such a place soon, the person you know who lives there. 460 00:37:11,755 --> 00:37:14,455 So if you buy a present now, you'll be able to take it with you if you go there. 461 00:37:14,455 --> 00:37:17,835 But if you're not going there, then maybe you need to post something now. 462 00:37:17,835 --> 00:37:18,690 It's like... 463 00:37:18,690 --> 00:37:27,442 those really nice little features, which again, AI assistant, well, know, Alexa just wasn't have had like, you know, four years ago until LMS arrived. 464 00:37:27,442 --> 00:37:29,573 think that tech was just so out of reach for them. 465 00:37:29,573 --> 00:37:31,944 I think bringing it in now is such a thing. 466 00:37:31,944 --> 00:37:39,736 And again, it starts to sort of almost like tip the balance maybe in the future as clients are going, well, actually AI is really actually quite useful for me at home. 467 00:37:39,736 --> 00:37:42,617 Like I'm finding so much utility into it. 468 00:37:42,617 --> 00:37:47,538 So actually maybe I'm not going to be so concerned about the legal firms usage. 469 00:37:48,280 --> 00:37:48,991 Yeah. 470 00:37:48,991 --> 00:38:00,401 The, uh, you know, and right now I feel like there's such a, an explosion of exploration that's happening with AI in, in the legal world. 471 00:38:00,401 --> 00:38:12,623 I have a friend of mine who he had a, I don't know the details of his exit, but I, I know enough to know he did pretty well and he started another company that is AI focused and 472 00:38:12,623 --> 00:38:13,750 he's older than me. 473 00:38:13,750 --> 00:38:14,218 I, 474 00:38:14,218 --> 00:38:16,010 first thing I asked him was like, what the hell are you doing, man? 475 00:38:16,010 --> 00:38:16,890 Go enjoy your money. 476 00:38:16,890 --> 00:38:17,961 Why are you doing this again? 477 00:38:17,961 --> 00:38:19,742 Why you putting yourself through this? 478 00:38:19,903 --> 00:38:29,241 But anyway, he created a solution that is AI focused and it could be used outside of legal. 479 00:38:29,241 --> 00:38:31,715 He just happens to have his Rolodex in legal. 480 00:38:31,715 --> 00:38:33,374 he started there. 481 00:38:33,494 --> 00:38:43,232 Well, he said that even when he has executive sponsorship, like the managing partner of the firm who wants to evaluate his platform, 482 00:38:43,266 --> 00:38:59,058 He is now getting put into a queue and a prioritization committee is managing when the firm will take a look because they have so many POCs and pilots in AI right now that 483 00:38:59,058 --> 00:39:00,419 they're being overwhelmed. 484 00:39:00,419 --> 00:39:10,506 like, I don't know, do you have any thoughts on how firms balance like the exploration piece with sustainable solutions in the longterm? 485 00:39:11,021 --> 00:39:21,601 Yeah, it's a really interesting one because I think that the whole sort of big shifts I feel like in AI sort of allow legal teams to really prototype ideas quickly without 486 00:39:21,601 --> 00:39:27,201 needing like dev team resources like myself, like, you know, do you need sentiment analysis on case data? 487 00:39:27,201 --> 00:39:29,081 Cool, AI will do that in a second. 488 00:39:29,121 --> 00:39:36,801 You want to test contract clause extraction and detection, know, AI can validate if that's going be useful or not before you need a full automation. 489 00:39:36,881 --> 00:39:38,437 But yeah, I feel like 490 00:39:38,862 --> 00:39:50,062 A lot of the problem is there's so many AI vendors at the moment, they've all got potentially really good tools, built predominantly on lots of foundation models, but it's 491 00:39:50,062 --> 00:40:00,742 that issue that we have around benchmarking is a lot of firms aren't necessarily in a position to be able to say, how do we test this effectively before we invest in this? 492 00:40:01,382 --> 00:40:07,149 And like you saying about your friend there, some vendors who are really focusing on the legal market could 493 00:40:07,149 --> 00:40:09,229 quite easily pivot away from legal. 494 00:40:09,669 --> 00:40:13,049 you could, know, know, selling to live law firms is slow. 495 00:40:13,209 --> 00:40:21,909 you know, procurement takes a long time, there's a lot of risk and regulatory requirements and you know, if the adoption is slow, then AI startups are probably just going to shift 496 00:40:21,909 --> 00:40:26,009 their focus entirely, like finance or compliance or healthcare. 497 00:40:26,369 --> 00:40:34,549 And I think maybe sometimes law firms are looking at it going, well, because we're slow, the vendors are pivoting and are they actually going to support our legal workflows going 498 00:40:34,549 --> 00:40:35,569 forward? 499 00:40:35,909 --> 00:40:37,005 Or, you know, 500 00:40:37,005 --> 00:40:38,705 Would there be an acquisition? 501 00:40:38,985 --> 00:40:46,985 know, cause you know, we saw a Thomson Reuters picking up so many companies very low as a case text for $650 million or so. 502 00:40:47,045 --> 00:40:55,305 And if you think, well, I was a case, you know, I wasn't, if I was a case test customer, like it's now just part of the Thomson Reuters Code Council product to say, do I want 503 00:40:55,305 --> 00:40:55,705 that? 504 00:40:55,705 --> 00:40:56,745 I didn't really want that. 505 00:40:56,745 --> 00:40:58,505 I just wanted what case texts were offering. 506 00:40:58,505 --> 00:41:02,949 it's, you know, if that tool gets rolled into a larger platform, you might not want it. 507 00:41:03,053 --> 00:41:09,853 Pricing can change what was previously a nice flexible standalone tool becomes a big enterprise licensing bundle. 508 00:41:10,453 --> 00:41:15,513 you know, it's really interesting that piece around like AI solutions appearing. 509 00:41:15,513 --> 00:41:17,953 are going, oh, cool, it's really got benefit in legal. 510 00:41:17,953 --> 00:41:23,013 But actually a lot of the benefit in legal with the AI stuff is applicable to so many industries. 511 00:41:23,013 --> 00:41:28,053 Like a lot of industries, a lot of them are just dealing with documents in the same way that legalists. 512 00:41:28,053 --> 00:41:29,613 We need get information out of documents. 513 00:41:29,613 --> 00:41:31,190 We need to categorize them. 514 00:41:31,190 --> 00:41:32,361 We need to get our insights. 515 00:41:32,361 --> 00:41:33,592 We need to do reporting. 516 00:41:33,592 --> 00:41:35,434 We need to do email summarisation. 517 00:41:35,434 --> 00:41:39,027 So legal is a really high value market to AMA. 518 00:41:39,027 --> 00:41:43,561 And there's a lot of lawyers who want to make use of this and get more efficiency out of their work. 519 00:41:43,561 --> 00:41:45,984 But there's so many other industries who want the same. 520 00:41:45,984 --> 00:41:53,540 And if they're willing to adopt quicker, then legal may find itself stuck without as many tools as it thought it may well have had originally. 521 00:41:53,752 --> 00:41:55,303 Well, and that's exactly what he's doing. 522 00:41:55,303 --> 00:42:01,909 He's pivoting away from legal because he's stuck in the mud at all these firms. 523 00:42:01,909 --> 00:42:02,760 You know, it's interesting. 524 00:42:02,760 --> 00:42:06,413 know, the pendulum, I feel like it's swinging back in the other direction. 525 00:42:06,413 --> 00:42:08,775 I've actually, it's incredible. 526 00:42:08,775 --> 00:42:18,043 The number of emails from VCs that I get, you know, wanting to get to know, you know, they see our growth on LinkedIn. 527 00:42:18,043 --> 00:42:21,856 I'm assuming, I don't know how else we would hit the radar. 528 00:42:21,996 --> 00:42:24,889 I could spend 40 hours a week just meeting with VCs. 529 00:42:24,889 --> 00:42:26,560 So I don't, I don't meet with any of them. 530 00:42:26,560 --> 00:42:30,373 We have one investor TLTF and like, that's all. 531 00:42:30,373 --> 00:42:31,894 It just doesn't make sense. 532 00:42:31,894 --> 00:42:45,546 But I have had a few casual conversations and what I'm hearing now is from also investors who feel maybe a little overexposed on with their portfolio on AI. 533 00:42:45,546 --> 00:42:48,648 You know, it went so hard, so fast. 534 00:42:48,648 --> 00:42:51,040 And now they're looking for companies that 535 00:42:51,040 --> 00:43:02,216 aren't necessarily gen AI exclusive and, who have, can demonstrate growth and good product market fit. 536 00:43:02,216 --> 00:43:13,122 So now we're seeing an interesting, again, the pendulum is swinging back a little bit where you got companies that are pivoting to, to markets where they can actually get some 537 00:43:13,122 --> 00:43:18,005 traction because there's such a glut of tools, point solutions out there. 538 00:43:18,005 --> 00:43:19,566 You've got investors. 539 00:43:19,566 --> 00:43:26,289 who are now looking for firms that AI isn't the central story to their narrative. 540 00:43:26,289 --> 00:43:28,520 So, you know, it's going to be interesting. 541 00:43:28,520 --> 00:43:38,724 And then combine that with, you know, there are some companies out there with some really big valuations where I don't think their revenue is nearly as stable. 542 00:43:38,724 --> 00:43:42,155 Yeah, they have great growth, but how much churn are they going to have? 543 00:43:42,155 --> 00:43:47,087 Because I hear stories off the record, like, yeah, this isn't actually working that great. 544 00:43:47,087 --> 00:43:48,278 And it's really expensive. 545 00:43:48,278 --> 00:43:49,068 And 546 00:43:49,198 --> 00:43:49,838 I don't know. 547 00:43:49,838 --> 00:43:54,566 Do you, do you feel like the pendulum is swinging back a smidge or, or no. 548 00:43:54,699 --> 00:44:04,839 Yeah, it's a funny one because I always think back to when I see all these huge valuations, know, prepared to what they're earning, I always sort look at Uber and how 549 00:44:04,839 --> 00:44:13,739 Uber integrated itself and like, you know, the taxes were so cheap back in the day, know, Uber was backed by so much VC money and then the VC went, well, hold on, where's my return 550 00:44:13,739 --> 00:44:14,459 here? 551 00:44:14,459 --> 00:44:18,419 And now Uber's are suddenly like, seemingly like 10 times the cost where I live. 552 00:44:18,419 --> 00:44:23,465 I'm like, oh, actually it's not really the great value I once thought it was because, know, VC have gone, I want my money. 553 00:44:23,465 --> 00:44:32,909 now kind of thing and it'd be interesting to see like you know will they be able to struggle, will they struggle to sort of maintain that I think especially if you're looking 554 00:44:32,909 --> 00:44:41,512 at tools where they're really targeting sort of prestige markets you know there's only so many big law firms around like to sort of have that and you know get them out of seats 555 00:44:41,512 --> 00:44:50,920 that you need whereas actually the middle market and the sort of smaller firms you know there's thousands or millions of those across the world so I 556 00:44:50,920 --> 00:44:52,971 Will they start targeting those instead? 557 00:44:52,971 --> 00:44:59,504 And obviously that then increases your sort of how much support you may have to do and then further eroding how much profit you can make. 558 00:44:59,504 --> 00:45:05,846 So yeah, it's, it's interesting to see how these sort of like really big legal, legal AI tools. 559 00:45:05,846 --> 00:45:10,678 It's like, I'm interested to see how they make their money back based on these valuations. 560 00:45:10,872 --> 00:45:22,889 Yeah, because investors in the VC world, they have to hit home runs one out of 10 times where the numbers just don't work. 561 00:45:22,889 --> 00:45:32,664 And those home runs, I if you look at Harvey at a $3 billion valuation, they're really too big to be acquired at this point, right? 562 00:45:32,664 --> 00:45:37,667 So it's a go public or bust scenario from an investor perspective. 563 00:45:37,667 --> 00:45:39,874 And is there enough of a TAM? 564 00:45:39,874 --> 00:45:42,536 you know, a total addressable market for that to happen. 565 00:45:42,536 --> 00:45:47,554 I don't, I'm not pretending to have the answer and they've got some incredible investors in their cap table. 566 00:45:47,554 --> 00:45:51,141 I mean, Google ventures, open AI, a 16 Z. 567 00:45:51,141 --> 00:45:59,656 I'm assuming they've done the math writing these big checks, but from an outsider, man, I'm having a hard time connecting the dots on how is this going to work? 568 00:46:00,490 --> 00:46:03,670 Yeah, it's crazy when talk about it, right? 569 00:46:03,670 --> 00:46:06,730 was like, well, you know, surely there can't be that much demand for it. 570 00:46:06,730 --> 00:46:10,450 But, know, it seems like there must be for the valuations that they're getting. 571 00:46:10,450 --> 00:46:14,070 Because like you say, they've got incredibly smart people working there. 572 00:46:14,070 --> 00:46:19,930 know, we've got, know, PhD students, PhD graduates, guess is what you call them. 573 00:46:19,930 --> 00:46:28,410 They have really incredible sort of legal technology people there beyond just like, oh, well, here's a thin wrapper around, you know, chat GBT, which a lot of the tools were very 574 00:46:28,410 --> 00:46:28,809 early on. 575 00:46:28,809 --> 00:46:33,549 that's all you need to do and you really need to differentiate you know now. 576 00:46:33,709 --> 00:46:45,409 So again it'll be interesting to see how that evolves over time and again like say with your friend like do they need to pivot into other markets going forward because again the 577 00:46:45,409 --> 00:46:55,571 issues that legal face are issues that other markets face and again can they break into like GCs again you only know maybe one or two licenses in every GC but there are 578 00:46:55,571 --> 00:46:57,241 millions of firms around the world. 579 00:46:57,241 --> 00:47:00,069 We've all got a couple of lawyers working there so yeah. 580 00:47:01,080 --> 00:47:02,280 Yeah, for sure. 581 00:47:02,280 --> 00:47:14,764 think that's definitely, I don't know how it's kind of a little bit like if you're selling to law firms, but you're also selling to GCs and making them more self-sufficient, you're 582 00:47:14,764 --> 00:47:17,585 taking away from law firm revenues. 583 00:47:17,585 --> 00:47:22,036 So I don't know how you straddle that fence, but they seem to be doing well. 584 00:47:23,256 --> 00:47:24,237 I wish them the best. 585 00:47:24,237 --> 00:47:29,272 want to see, I want to see a success, especially from, you know, which is 586 00:47:29,272 --> 00:47:31,303 what is arguably the leader in the space. 587 00:47:31,303 --> 00:47:41,183 Certainly they've got, you don't see, you know, I've been around the space for a long time, almost 20 years in legal tech and you don't see investors like that in legal tech 588 00:47:41,183 --> 00:47:42,975 cap tables, right? 589 00:47:42,975 --> 00:47:43,935 Like ever. 590 00:47:43,935 --> 00:47:45,617 I've never seen that. 591 00:47:45,617 --> 00:47:49,020 So I, I want, you know, the investment is good. 592 00:47:49,020 --> 00:47:50,601 I want the investment to keep flowing. 593 00:47:50,601 --> 00:47:51,692 So I wish them the best. 594 00:47:51,692 --> 00:47:56,136 It's just, uh, yeah, I, they know some things I don't. 595 00:47:56,873 --> 00:48:06,293 Yeah, mean, again, they must have incredible, incredibly persuasive talents, which again, I think is where maybe a lot of other legal tech has failed in the past because maybe 596 00:48:06,293 --> 00:48:15,013 you've had start-up series, know, it's a, know, probably maybe a developer and a lawyer and they're going, you know, we really focused on the tool, but actually, like you say, 597 00:48:15,013 --> 00:48:20,168 for a lot of things you see, it's not how amazing the tool is initially, it's how well you can market it how, you know. 598 00:48:20,168 --> 00:48:27,828 how well your contacts are in getting into these big firms and being able to talk to people and convincing them this is where they should be going going forward. 599 00:48:28,128 --> 00:48:38,068 Especially when you think of how slow law firms work and how slowly they move in terms of adoption of tools, being able to convince them to adopt your tool and adopt it quickly is 600 00:48:38,068 --> 00:48:39,288 an incredible talent. 601 00:48:39,288 --> 00:48:44,288 And I wish I had as much of that trying to get some internal tool adoption at times. 602 00:48:44,354 --> 00:48:46,115 Yeah, absolutely. 603 00:48:46,115 --> 00:48:47,716 Well, we're, we're about out of time. 604 00:48:47,716 --> 00:48:49,097 This has been a great conversation. 605 00:48:49,097 --> 00:48:55,540 I know we, uh, we, kinda went off script a little bit with the KPMG stuff, but it's, it's so timely. 606 00:48:55,540 --> 00:49:04,045 I'm actually recording a podcast tomorrow with Debbie Foster from affinity consulting, and we're going to talk about the big fours push. 607 00:49:04,045 --> 00:49:06,156 Um, yeah, I'm a, I'm not a legal guy. 608 00:49:06,156 --> 00:49:07,116 I don't have a JD. 609 00:49:07,116 --> 00:49:12,369 I've got an MBA and I'm, I consider myself kind of a legal market analyst. 610 00:49:12,369 --> 00:49:14,470 Um, and I, 611 00:49:15,195 --> 00:49:18,017 All of these competitive dynamics really interest me. 612 00:49:18,017 --> 00:49:25,332 So yeah, it's going to be an interesting next couple of years to see, to see how this plays out. 613 00:49:25,596 --> 00:49:31,215 Yeah, I'm going to have a very busy time building out more tooling to support our US firm, so it's going to be great fun. 614 00:49:31,382 --> 00:49:33,686 Yeah, yeah, you got good job security. 615 00:49:33,686 --> 00:49:39,574 All right, well, I really appreciate you joining this afternoon and let's definitely stay in touch. 616 00:49:39,912 --> 00:49:42,082 Yeah, much appreciated, speak to you soon. 617 00:49:42,082 --> 00:49:43,785 All right, thanks, Ryan. 00:00:04,853 Ryan McDonough, how are you this afternoon? 2 00:00:05,250 --> 00:00:07,070 Very good, thank you, how are you Ted? 3 00:00:07,278 --> 00:00:07,749 I'm doing good. 4 00:00:07,749 --> 00:00:10,976 Actually, I guess it's good evening where you are, right? 5 00:00:11,230 --> 00:00:17,723 Indeed, despite the very bright background, it's incredibly dark outside, proper midst of winter here at the moment. 6 00:00:17,794 --> 00:00:18,834 Gotcha. 7 00:00:18,834 --> 00:00:22,956 Well, I have enjoyed your posts on LinkedIn. 8 00:00:22,956 --> 00:00:28,877 You know, I talk a lot about AI because legal innovation is really kind of my focus. 9 00:00:28,877 --> 00:00:32,798 And obviously AI is a big part of that equation. 10 00:00:32,798 --> 00:00:38,720 But, and I think we got a really good agenda for today, but before we jump in, let's get you introduced. 11 00:00:38,760 --> 00:00:42,981 You are currently the head of software engineering at KPMG law. 12 00:00:43,201 --> 00:00:45,382 You also spent some time in legal. 13 00:00:45,972 --> 00:00:49,825 as a software team lead at AG. 14 00:00:50,082 --> 00:00:54,599 Why don't you tell us a little bit about where you're at today and a little bit about your background. 15 00:00:55,005 --> 00:00:56,336 Yeah, sure. 16 00:00:56,336 --> 00:01:05,673 know, prior to sort of working in law, I was working in digital agencies and other small companies, moved into law because there was a really interesting role that I saw, which 17 00:01:05,673 --> 00:01:13,479 was to lead an innovation team, which was a brand new concept that I thought I'd never seen in law, like, you know, trying to make law sexy, which was, which was quite a hard 18 00:01:13,479 --> 00:01:15,901 task, but it's quite an easy one now. 19 00:01:15,941 --> 00:01:18,433 And yeah, so I saw a role like, this, sounds great. 20 00:01:18,433 --> 00:01:20,937 And it was really focusing on how do we... 21 00:01:20,937 --> 00:01:28,830 improve how we deliver to clients and how do we improve how our legal teams work internally and is that building software tools, that just improvement processes? 22 00:01:28,830 --> 00:01:32,541 So it felt like a really good move to move into that position. 23 00:01:32,822 --> 00:01:38,444 And then I moved into KPMG, again focusing on global solutions. 24 00:01:38,444 --> 00:01:48,848 So we're trying to figure out where do we put the most time in put in our priority solutions which have the most effects across all of our member firms across the world. 25 00:01:49,459 --> 00:01:56,875 So again, lot of the time at the moment, as you can imagine, is spent on AI and trying to figure out where to replace this most tactfully in the most useful manner without just 26 00:01:56,875 --> 00:02:00,372 splashing it around like really nearly, I suppose. 27 00:02:01,536 --> 00:02:02,116 Interesting. 28 00:02:02,116 --> 00:02:02,396 Yeah. 29 00:02:02,396 --> 00:02:18,093 KPMG is a frequent topic of conversation here in the States with their recent approval that somehow made it all the way its way to the Supreme court of Arizona to set up shop as 30 00:02:18,093 --> 00:02:19,364 an alternative business structure. 31 00:02:19,364 --> 00:02:25,676 Um, and yeah, I did a presentation at a conference recently. 32 00:02:25,676 --> 00:02:30,092 In fact, it was just three days ago and talked a little bit about 33 00:02:30,092 --> 00:02:38,609 the state of legal innovation and some of the challenges that law firms face truly innovating. 34 00:02:38,629 --> 00:02:52,301 And, you know, I'm a believer that today most of what we're calling innovation in the legal world and the law firm world is really more improvement. 35 00:02:52,321 --> 00:02:58,166 you know, the difference being improvement is incremental and focusing on efficiency and 36 00:02:58,166 --> 00:03:04,139 innovation is really disruptive and kind of rethinking business models and processes. 37 00:03:04,139 --> 00:03:06,351 And not a lot of that has happened in legal. 38 00:03:06,351 --> 00:03:11,654 There is some for sure, but, um, it's been, it's, it's been a lot of improvement. 39 00:03:11,654 --> 00:03:26,424 And, know, I made the case in this presentation that entities like KPMG, which is the smallest of the big four, but still 40 billion in revenue, 4,000 attorneys. 40 00:03:27,061 --> 00:03:42,991 275,000 employees, a global footprint, legions of digital transformation experts and gen AI engineers and Lean Six Sigma black belts and physical assets like the ignition center 41 00:03:42,991 --> 00:03:55,188 in Denver, where you guys bring together clients and consultants from tax audit and advisory and solve problems in a space custom designed for that. 42 00:03:55,192 --> 00:03:57,563 We don't see a lot of that in law firms. 43 00:03:57,563 --> 00:03:59,502 And I did a survey. 44 00:03:59,502 --> 00:04:14,130 I asked after I went through and listed all of those attributes of KPMG and then highlighted that almost two thirds, 63 % of law firm clients either disagree or strongly 45 00:04:14,130 --> 00:04:16,851 disagree that their law firm is innovative. 46 00:04:17,311 --> 00:04:23,874 And even though there's been almost 2000 % growth in the innovation function in the last 10 years, 47 00:04:25,100 --> 00:04:29,652 still almost two thirds of clients don't feel that their law firms are innovative. 48 00:04:29,652 --> 00:04:38,375 And I attribute a lot of that to innovation theater where law firms do things to look innovative, but don't fundamentally commit to change. 49 00:04:39,376 --> 00:04:44,018 so yeah, I, I, think this is a big moment right now. 50 00:04:44,018 --> 00:04:53,562 And I, know, the big four have pushed in, in the U S before with, you know, moderate success, but I think this time is different, different because the means of legal 51 00:04:53,562 --> 00:04:54,582 production, 52 00:04:54,710 --> 00:04:56,233 are really changing with Gen.ai. 53 00:04:56,233 --> 00:04:59,198 So I don't know, what's your take on all that? 54 00:04:59,933 --> 00:05:10,967 Yeah, it's an interesting one because yeah, you say that the innovation theatre and it's definitely something that I sort of see a lot of because you always need to be seen to 55 00:05:10,967 --> 00:05:21,181 continually innovate in an innovation team and it's really hard because there's those tensions between, you know, what is being truly innovative and really changing how the 56 00:05:21,181 --> 00:05:27,103 market works for your law firm and what is doing the bread and butter stuff that needs to be done every day to keep the firm running. 57 00:05:27,103 --> 00:05:28,957 It's how do you decide what to 58 00:05:28,957 --> 00:05:31,948 really invest in when there's no fallback. 59 00:05:31,948 --> 00:05:40,332 And I think that's where KPMG can differ in that sense because like you say, we've got so many teams who do so many incredible things. 60 00:05:40,332 --> 00:05:46,125 It really reminded me of when I moved to a digital agency, were writing more front-end or back-end. 61 00:05:46,125 --> 00:05:48,096 was like, I'm pretty good at both. 62 00:05:48,096 --> 00:05:52,232 And then when I saw what the front-end developers could do there, was like, yeah, I'm probably more back-end. 63 00:05:52,232 --> 00:05:54,783 Like those guys are rock stars at what they do. 64 00:05:54,783 --> 00:06:02,863 And it's the same, like when you think of the ignition center, the UX experts, the AI experts, it's like, I'm really into AI, but there some people are so deep into it. 65 00:06:02,863 --> 00:06:03,583 It's incredible. 66 00:06:03,583 --> 00:06:09,463 And having that expertise available that, you can just go, you know, for this project, we need this. 67 00:06:09,463 --> 00:06:10,543 They'll cross charge. 68 00:06:10,543 --> 00:06:17,423 We get that expertise and that's available to us in a way that, you know, if you're maybe a traditional waterfront, like, we need an expert. 69 00:06:17,423 --> 00:06:17,723 Okay. 70 00:06:17,723 --> 00:06:21,063 Well, let's put a contract out, you know, see if we can get some other three months. 71 00:06:21,063 --> 00:06:23,103 Have you got a business case for this? 72 00:06:23,103 --> 00:06:27,323 you know, who's going to keep an AI staff on staff for a full year. 73 00:06:27,323 --> 00:06:33,823 No one, but actually this KPMG can have that person work on client projects and audit projects and consulting projects. 74 00:06:33,903 --> 00:06:37,183 They have a reason to be there all the time and we can dip into that resource. 75 00:06:37,183 --> 00:06:40,723 I think that's why it's really powerful to have those people on hand. 76 00:06:41,134 --> 00:06:41,574 Yeah. 77 00:06:41,574 --> 00:06:43,374 And you you touched on something. 78 00:06:43,374 --> 00:06:50,914 Uh, so the amount of friction in law firm decision-making is stifling to innovation, right? 79 00:06:50,914 --> 00:06:55,074 Everything is a committee and you've got owners of the business. 80 00:06:55,074 --> 00:07:07,154 And this is true to a certain extent with the big four, the big four are partnership models as well, but they have much less friction and they operate closer to what you would 81 00:07:07,154 --> 00:07:10,414 see in a traditional fortune 500 company. 82 00:07:10,476 --> 00:07:11,736 And I know a little bit about this. 83 00:07:11,736 --> 00:07:17,928 Like I've seen it kind of from the outside, but I spent 10 years at Bank of America and we worked closely with the big four. 84 00:07:17,928 --> 00:07:23,010 was in, um, I was in corporate audit, anti-money laundering, consumer risk. 85 00:07:23,010 --> 00:07:27,621 So we were engaging with the big four all the time in various capacities and. 86 00:07:27,621 --> 00:07:32,672 know, decision-making happens differently there than it does in law firms. 87 00:07:32,672 --> 00:07:39,014 And you guys do have those resources that traditional law firms don't and. 88 00:07:39,416 --> 00:07:41,727 things are going to change so fast. 89 00:07:41,788 --> 00:07:47,292 Innovation is in your culture at KPMG, like true innovation, right? 90 00:07:47,292 --> 00:07:49,453 It's not in Big Law. 91 00:07:49,513 --> 00:07:58,639 And I think they have aspirations of wanting innovation to be integral to their culture, but they're not there today. 92 00:07:58,640 --> 00:08:05,035 So I think the Big Four has some advantages going in. 93 00:08:05,035 --> 00:08:07,680 Big Law has some advantages too, but 94 00:08:07,680 --> 00:08:12,533 It's going to be interesting to see how this tension plays out in the marketplace. 95 00:08:13,149 --> 00:08:14,249 Oh yeah, definitely. 96 00:08:14,289 --> 00:08:22,789 like I said, think it all, a lot of it just comes back to this, you know, having those people available means that, you know, they can do so much other work that, you you just 97 00:08:22,789 --> 00:08:23,609 can't do it all. 98 00:08:23,609 --> 00:08:29,029 You can't go, oh, well we can, you know, get this AI expert to consult with clients and we'll get some billable time off them. 99 00:08:29,029 --> 00:08:32,089 It's just not really feasible having two people doing that. 100 00:08:32,089 --> 00:08:38,669 Whereas we have that capability and we have that interesting building fast and building well. 101 00:08:38,669 --> 00:08:42,769 And we have enough resource to be able to do that at scale, which is incredible. 102 00:08:42,973 --> 00:08:50,733 Again, there are projects underway doing huge AI infrastructure globally, which is needed for a firm like APMG. 103 00:08:50,733 --> 00:08:52,933 And to a far, I think it's needed for most law firms. 104 00:08:52,933 --> 00:09:04,433 If you're of a reasonable size, having an AI architecture in place that would allow business teams to be able to consume it through power automate and power apps in a safe 105 00:09:04,433 --> 00:09:09,273 way that they know is being audited, they know is being have correct guardrails around it. 106 00:09:09,273 --> 00:09:11,709 It's really valuable to have that available. 107 00:09:11,709 --> 00:09:17,609 in a way that means they don't have to spend all the time reinventing the core capability that everyone needs for AI. 108 00:09:17,609 --> 00:09:19,249 We all need auditing. 109 00:09:19,249 --> 00:09:23,129 We all need to log how many tokens are being used for cost effectiveness. 110 00:09:23,969 --> 00:09:28,829 And having that as an AI architecture that just anyone can delve into is so impressive. 111 00:09:28,829 --> 00:09:31,329 It can be a GMG, I can just use this AI capability. 112 00:09:31,329 --> 00:09:35,929 I don't have to spend time doing that core, the lower the water level iceberg work. 113 00:09:35,929 --> 00:09:38,949 I can work on the actual stuff that provides impact. 114 00:09:38,949 --> 00:09:40,167 Whereas I find out a lot of law firms. 115 00:09:40,167 --> 00:09:41,638 they have to reinvent that piece. 116 00:09:41,638 --> 00:09:47,360 They have to go out and find a vendor to offer that piece and you've got to then, you know, spend time and money and effort doing that. 117 00:09:47,360 --> 00:09:50,731 it's just, again, there's no necessary ROI on that. 118 00:09:50,731 --> 00:09:57,143 Whereas I KPMG looks at us, we can see the ROI coming for this and we can absorb this for the next year or two years. 119 00:09:57,143 --> 00:10:00,604 And so we see that ROI arriving for us. 120 00:10:00,846 --> 00:10:01,746 Yeah. 121 00:10:01,746 --> 00:10:11,686 And you know, I think part of the, part of the challenge in the U S too is, and it's not just the U S I think globally in the law firm world is it's very fragmented. 122 00:10:11,686 --> 00:10:16,306 So you have the big four and then you have the AMLAL 200, right? 123 00:10:16,306 --> 00:10:26,946 It's like you've got, so you've got four accounting firms that range from 40 at the low end with KPMG to Deloitte, which is 80 billion. 124 00:10:27,146 --> 00:10:27,648 Um, 125 00:10:27,648 --> 00:10:32,419 And then you've got the biggest law firm, which is a bit of an outlier, which is Kirkland and Ellis. 126 00:10:32,419 --> 00:10:35,910 That's about 7.2, 7.3 billion in revenue. 127 00:10:35,910 --> 00:10:40,732 So KPMG is almost six times the biggest law firm and they're the smallest of the big four. 128 00:10:40,732 --> 00:10:47,604 you know, having those resources is a huge advantage. 129 00:10:47,604 --> 00:10:53,165 And I think an even bigger advantage though, is just the cultural aspects. 130 00:10:53,265 --> 00:10:55,026 It's, it really is hard. 131 00:10:55,026 --> 00:10:56,806 You have to establish. 132 00:10:57,550 --> 00:11:07,496 culture at the top of the organization and at the top of the organization of many law firms are executive committees that are, it's an older demographic typically. 133 00:11:07,496 --> 00:11:08,986 And I've said this a million times. 134 00:11:08,986 --> 00:11:17,501 My listeners are probably tired of hearing it, but a lot of those leaders got there by being the best at lawyering, not because they were necessarily the best leader. 135 00:11:17,721 --> 00:11:18,322 Right. 136 00:11:18,322 --> 00:11:23,284 And I think the decision-making happens differently within the big four. 137 00:11:23,284 --> 00:11:26,794 It's not that you were the best tax auditor. 138 00:11:26,794 --> 00:11:31,762 why you're leading the charge of the tax practice. 139 00:11:31,762 --> 00:11:37,740 But yeah, I think that's another dimension where Big Four has an advantage. 140 00:11:38,172 --> 00:11:41,212 Yeah, think, I think our culture putting mate is really big. 141 00:11:41,212 --> 00:11:51,072 mean, like every Friday, there's, mean, just, in our UK community, there's an AI call that happens every Friday where people are sharing what they've seen in AI and that's across, 142 00:11:51,072 --> 00:11:54,032 you know, consulting, it tags and legal. 143 00:11:54,052 --> 00:11:58,352 We're all chatting about what we've been doing, the ideas we've seen, helping people digest. 144 00:11:58,352 --> 00:12:05,272 Because I always think it's amazing when I see like just how much AI research and like what's the KPMG are putting out. 145 00:12:05,272 --> 00:12:06,392 Who's doing all this? 146 00:12:06,392 --> 00:12:07,752 Like it's crazy. 147 00:12:08,380 --> 00:12:16,360 how big these teams are, that there's just a team dedicated to writing these reports and having like those people just going around sharing stuff like going, oh, cool, I didn't 148 00:12:16,360 --> 00:12:16,880 know that existed. 149 00:12:16,880 --> 00:12:19,260 I can have a look at that and take that. 150 00:12:19,260 --> 00:12:22,120 And there is like a big enough culture to achieve that. 151 00:12:22,120 --> 00:12:24,500 I mean, there's huge networks in KPMG. 152 00:12:24,500 --> 00:12:29,440 it's just like, there's a whole football network and it's just, there's enough people to warrant having that. 153 00:12:29,440 --> 00:12:36,620 And again, other law firms, there's maybe not enough groundswell to build out an AI network or a football network. 154 00:12:36,620 --> 00:12:38,053 So I think it's... 155 00:12:38,053 --> 00:12:48,037 Having that share massive people also helps innovation because you've just got so many ideas bouncing around and so many different viewpoints from different cultures just really 156 00:12:48,037 --> 00:12:50,229 helps spur on more innovation I find. 157 00:12:50,594 --> 00:12:50,864 Yeah. 158 00:12:50,864 --> 00:12:52,575 And you guys have infrastructure as well. 159 00:12:52,575 --> 00:13:06,679 Like I was reading about the KPMD digital gateway that I think started on the tax side of the business, but is now a gateway for some of your legal applications as well. 160 00:13:06,679 --> 00:13:07,840 Is that correct? 161 00:13:07,995 --> 00:13:09,035 That's correct, yeah. 162 00:13:09,035 --> 00:13:11,995 So it started off as a digital gateway for tax. 163 00:13:11,995 --> 00:13:14,295 We've just got digital gateway at that point. 164 00:13:14,295 --> 00:13:16,935 But now we have the digital gateway for law. 165 00:13:16,935 --> 00:13:23,175 So it's basically it's like a personalized entry point for accessing all the KPMG technology and services. 166 00:13:23,235 --> 00:13:29,595 So, you know, if you're consuming hopefully many different services from KPMG, they're all in one place. 167 00:13:29,595 --> 00:13:34,695 Again, with different levels of integration and performance that's included into them. 168 00:13:34,695 --> 00:13:36,579 But you know, like we've built out a... 169 00:13:36,579 --> 00:13:39,300 international business reorganization platform. 170 00:13:39,541 --> 00:13:41,362 Sometimes hard to say that quickly. 171 00:13:41,362 --> 00:13:45,705 And we can service information from that inside the digital gateway for more. 172 00:13:45,705 --> 00:13:48,907 So it means that, you know, the client can go onto digital gateway. 173 00:13:48,907 --> 00:13:52,179 They can see all the latest reports that KPMG have put out. 174 00:13:52,179 --> 00:13:55,592 They can see the horizon scanning and go, actually, how's my program? 175 00:13:55,592 --> 00:13:56,953 How's my project progressing? 176 00:13:56,953 --> 00:13:57,473 Click on that. 177 00:13:57,473 --> 00:13:59,394 I can see what's progressing there. 178 00:13:59,474 --> 00:14:00,175 And then I'm done. 179 00:14:00,175 --> 00:14:01,726 I don't have to go off to a different platform. 180 00:14:01,726 --> 00:14:03,837 I don't have to do 58 clicks to get there. 181 00:14:03,837 --> 00:14:04,928 It's all in one place. 182 00:14:04,928 --> 00:14:05,658 So 183 00:14:05,679 --> 00:14:09,761 we're really trying to focus it on that sort of like personalized entry point piece. 184 00:14:10,161 --> 00:14:14,814 And again, it's a really good place because again, there's so many developers working on digital gateway. 185 00:14:14,814 --> 00:14:22,348 So again, when we're looking at ideas of how can we do this and how can we expand, know, there's so many people working on Gen.AI for digital gateway. 186 00:14:22,348 --> 00:14:27,731 It's a really big focus they've got at the moment allowing the clients and our internal teams to use Gen.AI. 187 00:14:27,731 --> 00:14:34,464 So we're trying to bring all those capabilities that clients are talking about and asking for and giving it to them as well. 188 00:14:35,150 --> 00:14:35,640 It's interesting. 189 00:14:35,640 --> 00:14:41,355 So we at info dash started our journey on the internet side. 190 00:14:41,355 --> 00:14:43,467 So we're pretty well established there. 191 00:14:43,467 --> 00:14:49,561 Been doing the work as consultants for 14 years before we productized three years ago. 192 00:14:49,562 --> 00:14:51,443 So we've got, and we've got a great install base. 193 00:14:51,443 --> 00:15:05,314 We just launched an extranet platform, which sounds similar to your digital gateway and the benefit of our approach, because we also see scenarios where 194 00:15:05,599 --> 00:15:08,691 Law firm clients are going to want access. 195 00:15:08,691 --> 00:15:22,161 like the, the incumbent in vendor in the space is high Q high Q is a very ring fenced standalone hosted solution that operates completely outside of the law firms. 196 00:15:22,161 --> 00:15:23,782 Technology environment. 197 00:15:23,782 --> 00:15:25,884 We think there's a better way. 198 00:15:25,884 --> 00:15:34,286 So we have built info dash extra net that gets deployed in the law firms tenant and has access to. 199 00:15:34,286 --> 00:15:37,026 built in integrations with all the backend systems. 200 00:15:37,266 --> 00:15:48,446 So finance, you know, analytics, BI practice management, document management, experience management, CRM, HR, IS so we can pull together. 201 00:15:48,446 --> 00:15:57,586 So if you're a law firm client, you can log in, uh, to one of your matter sites and you can see the entire legal team that is billed time to your matter. 202 00:15:57,586 --> 00:16:02,306 You can see their experience, who they clerk for, what bars they're admitted to. 203 00:16:02,306 --> 00:16:03,362 You can see 204 00:16:03,362 --> 00:16:11,548 details about your engagement letters, any documents related to the matter, any dockets associated with the matter. 205 00:16:12,569 --> 00:16:24,578 So we are betting big on law firms wanting to compete with things like Digital Gateway so they can provide access because today it just doesn't happen. 206 00:16:24,578 --> 00:16:29,581 It's really hard to get information in and out of HiQ on purpose, right? 207 00:16:29,750 --> 00:16:34,002 There's a lot of, yeah, there's definitely a lot of data silos with a lot of legal tech tools. 208 00:16:34,002 --> 00:16:41,516 And I think that's just with the intention of retain clients because you simply can't get data in and out of them. 209 00:16:41,516 --> 00:16:50,661 And I think that the idea of having connections to all the systems in one place, because I think, I'm saying like, you'll be going full AI for everything, but it's having that 210 00:16:50,661 --> 00:16:55,724 going, AI would have the context of the whole issue for the client. 211 00:16:56,064 --> 00:16:57,675 What does the lawyer know about the client? 212 00:16:57,675 --> 00:16:59,536 What do we know from our horizon scanning? 213 00:16:59,536 --> 00:17:01,726 What do we know from all the other databases we have? 214 00:17:01,726 --> 00:17:04,508 What people have been talking about the client recently? 215 00:17:04,508 --> 00:17:06,358 And then that's all accessible at that point. 216 00:17:06,358 --> 00:17:15,604 So you have that, you know, that's canonical source of truth of the client and all their requirements rather than again, saying, well, some of this is in this system and some of 217 00:17:15,604 --> 00:17:16,753 this is in this system. 218 00:17:16,753 --> 00:17:21,215 actually we're going to have to do a huge uplift to integrate them all into like some sort of data wake. 219 00:17:21,215 --> 00:17:24,476 It makes sense to go down the approach that you are. 220 00:17:24,578 --> 00:17:26,039 Yeah, we think so. 221 00:17:26,039 --> 00:17:35,816 And we think over time, the competitive pressures in the landscape are really going to force both the vendors, the legal tech vendors and law firms to kind of reevaluate the 222 00:17:35,816 --> 00:17:39,688 current strategy, which is clients don't really get what they need today. 223 00:17:39,989 --> 00:17:47,334 Very few, very few clients even share financial data with their, with their customers through, through extra nets. 224 00:17:47,334 --> 00:17:49,765 Like maybe they have the last X number of bills. 225 00:17:49,765 --> 00:17:52,037 There's very little work in process. 226 00:17:52,037 --> 00:17:54,190 That data has to get massaged. 227 00:17:54,190 --> 00:18:04,013 Um, you know, time entry data before it gets, it gets released, uh, to, clients, but clients want transparency and they don't have it today. 228 00:18:04,013 --> 00:18:14,556 So yeah, we're putting our chips on the table saying, you know what, you've been able to get away with this because you just have big, big law has been very resilient over the 229 00:18:14,556 --> 00:18:15,486 last 50 years. 230 00:18:15,486 --> 00:18:15,906 Right. 231 00:18:15,906 --> 00:18:19,057 But the, think times are a changing. 232 00:18:19,057 --> 00:18:23,678 Um, I think there's going to be new competitive pressures that are going to 233 00:18:23,886 --> 00:18:26,259 cause some different ways of thinking. 234 00:18:27,064 --> 00:18:35,664 I think there's been that sort of push by what people's expectations are around the applications and even the billing piece. 235 00:18:35,664 --> 00:18:47,324 again, I remember when my first job was at a law firm many years ago and a partner turned up and went, turn this into a website, gave me a form and I need a website and I didn't 236 00:18:47,324 --> 00:18:51,004 see them for six months and they turned back up and they were like, wow, that's amazing. 237 00:18:51,004 --> 00:18:53,879 People are amazed that a website could exist back then. 238 00:18:53,879 --> 00:18:58,659 But, you know, people are so used to like the applications on the phones and the quality that they have. 239 00:18:58,659 --> 00:19:04,059 And again, being able to see the bills, they go, well, I can see my bill for Google suite and I can see my bill for Uber. 240 00:19:04,059 --> 00:19:06,159 So why can't I see my legal bills? 241 00:19:06,279 --> 00:19:13,479 And it's, and again, it's that, it's that thing of going, well, you know, if I'm requesting a bill off the legal team, that's time out of the legal team's day. 242 00:19:13,479 --> 00:19:16,439 You know, just responding to that particular request. 243 00:19:16,439 --> 00:19:18,039 I'm going, oh, where's my bill? 244 00:19:18,039 --> 00:19:19,819 Can you forward it on to me? 245 00:19:20,310 --> 00:19:27,916 non-billable time and it's you know, useless and it's taking you out of your contact switching, especially if it's like a really big client and they're, know, really important 246 00:19:27,916 --> 00:19:34,791 and you're going, oh, I have to that email, have to, you know, stop doing what I'm doing, get this bill from whatever system and send it off to them. 247 00:19:35,012 --> 00:19:43,168 Again, maybe if that person goes on holiday and then someone else wants that same request, it's putting the data in front of the client so they can self-serve as much as is possible 248 00:19:43,168 --> 00:19:49,323 because clients, I think, are getting more and more used to that concept now and I think they prefer it from lot of issues. 249 00:19:49,474 --> 00:19:51,495 Yeah, that totally makes sense. 250 00:19:51,495 --> 00:20:05,597 Well, you and I were going to talk a little bit about AI today and specifically we were going to talk about benchmarks and you know, I have conflicting feelings about the 251 00:20:05,597 --> 00:20:06,479 benchmarks. 252 00:20:06,479 --> 00:20:19,362 I feel like in this rush for the first AI vendor to plant their flag on Planet AGI that the benchmarks have really focused 253 00:20:19,362 --> 00:20:30,998 That's been a focal point and we see, you know, it can do all these PhD level tasks, but it still can't summarize my spreadsheet, my pivot table, you know, just the basics of, you 254 00:20:30,998 --> 00:20:36,050 know, blocking and tackling use cases are still very hit or miss. 255 00:20:36,210 --> 00:20:45,014 So like, what are you, what is your take on like the limitations of the current benchmarks, you know, for like real world legal applications? 256 00:20:45,207 --> 00:20:52,487 Yeah, it's interesting because like we say, I we've seen the VALS AI piece recently and that's a big step improvement on how it is. 257 00:20:52,487 --> 00:20:58,367 But I mean, a lot of the academic AI benchmarks just aren't designed for real world legal work. 258 00:20:58,367 --> 00:21:07,847 They're testing sort of AI on very isolated tasks like extracting clauses, summarizing documents and answering simple legal questions that maybe I could even have a go at. 259 00:21:07,847 --> 00:21:11,007 know, law isn't these disconnected tasks. 260 00:21:11,007 --> 00:21:12,502 It's about context. 261 00:21:12,502 --> 00:21:17,846 and reasoning and linking those different bits of information across all these different documents. 262 00:21:18,067 --> 00:21:22,350 it's, know, mean, like, you know, a model might be able to extract a termination clause. 263 00:21:22,402 --> 00:21:28,886 But in practice, like, the lawyers want to know, like, how does this interact with liability caps and notice periods and local regulations? 264 00:21:28,886 --> 00:21:32,970 And are there similar clauses that contradict it in the same document? 265 00:21:32,970 --> 00:21:36,083 And does it actually align with your client's risk profile? 266 00:21:36,083 --> 00:21:40,394 And that's where the academic benchmarks fall down because they're not measuring. 267 00:21:40,394 --> 00:21:47,299 that level of legal reasoning, document complexity and just the real world constraints that we face every day with documents. 268 00:21:47,299 --> 00:21:52,988 I mean, even when I was building AI systems very early on, when ChatGPT came out, the API was available. 269 00:21:52,988 --> 00:21:57,496 I was like, cool, I'll get this Word document that someone's made for me and I'll analyze that. 270 00:21:57,496 --> 00:22:04,251 And it was great because it was a Word document and it was exactly structured how I needed it to be, but that's not really reflective of the real world. 271 00:22:04,251 --> 00:22:08,954 It's not a handwritten annotations on a document and drawings and... 272 00:22:09,000 --> 00:22:14,392 literal red lines on the document from versions of it with signatures overlapping words. 273 00:22:14,392 --> 00:22:22,864 it's, know, as a human I go, well, that probably says director because I can see the word D-I-R and then O-R at the end of it, but part of the signature is going over it. 274 00:22:22,864 --> 00:22:25,615 But AI was like, don't know what that is. 275 00:22:25,615 --> 00:22:27,996 Very illiterate, I have no idea what that is. 276 00:22:27,996 --> 00:22:33,798 again, need, firms need to basically have their own benchmarks, I feel, to reflect their own workflows. 277 00:22:34,296 --> 00:22:37,987 that's how the conversation actually started. 278 00:22:38,927 --> 00:22:41,648 There is a legal benchmark out there. 279 00:22:41,648 --> 00:22:43,929 There's an associated GitHub repository. 280 00:22:43,929 --> 00:22:45,949 I saw the contributors. 281 00:22:45,949 --> 00:22:51,811 think of which, were you one or maybe you just pointed a link to that GitHub repository? 282 00:22:53,091 --> 00:22:53,572 Okay. 283 00:22:53,572 --> 00:23:02,554 Well, talk to us a little bit about developing custom benchmarks that better reflect legal use cases. 284 00:23:02,888 --> 00:23:03,618 Yeah, sure. 285 00:23:03,618 --> 00:23:11,863 So I feel like, you know, when you're trying to sort of move just beyond vendor demos, you know, where they're very highly polished and I've done this myself demoing tools 286 00:23:11,863 --> 00:23:15,744 internally, I have a very strict script I wanted to keep too. 287 00:23:16,225 --> 00:23:20,227 But when you want to try and build your own framework, you sort of base it on the real tasks. 288 00:23:20,227 --> 00:23:25,290 So sitting down with the team and figuring out what those are, it's like, you know, testing it on actual legal documents. 289 00:23:25,290 --> 00:23:31,669 So not just taking the ones that vendors are supplying you, it's finding the ones that are really reflective and go into the team and you know. 290 00:23:31,669 --> 00:23:34,109 What's been a weird document you've seen recently? 291 00:23:34,109 --> 00:23:40,989 Like I was trying to analysis backwards as an AG of a document, trying to do machine learning analysis of a document structure. 292 00:23:40,989 --> 00:23:50,429 And it was great and all that precedence that we had, but then I got a client document and the only way they differentiated between the section headers and the clauses was they used 293 00:23:50,429 --> 00:23:51,769 purple text. 294 00:23:51,809 --> 00:23:53,489 And that was the only differentiation. 295 00:23:53,489 --> 00:23:55,209 wasn't like one dot whatever. 296 00:23:55,209 --> 00:23:56,809 It was just a bit of purple text. 297 00:23:56,809 --> 00:23:58,429 And you're like, well, how do I train for that? 298 00:23:58,429 --> 00:23:59,709 That is crazy. 299 00:23:59,765 --> 00:24:07,205 But those are the kind of documents you want to use because you want to see if AI can actually handle the clients insane documents that have been handed over or just documents 300 00:24:07,205 --> 00:24:11,285 that you've got from 15 years ago that are actually materially relevant. 301 00:24:11,685 --> 00:24:14,945 And then you have to measure that impact. 302 00:24:14,945 --> 00:24:21,345 like, it reducing the time spent on review or is it just adding another layer of verification on top of it at the moment? 303 00:24:21,345 --> 00:24:26,684 And if it is just adding another layer of verification on top of it, again, making a note of that because again, 304 00:24:26,684 --> 00:24:27,915 That may change over time. 305 00:24:27,915 --> 00:24:35,814 may need to revisit these tools and the benchmarks that you have and go, well, actually this was previously something we had to verify, but now we don't have to verify it. 306 00:24:35,814 --> 00:24:37,453 So actually this is more applicable. 307 00:24:37,453 --> 00:24:42,468 And then again, evaluating the accuracy is so important in the real world conditions. 308 00:24:42,468 --> 00:24:48,443 It's, know, scan PDFs, handwritten notes, multiple jurisdictions, redacted clauses. 309 00:24:48,443 --> 00:24:52,226 How is it handling those within its system? 310 00:24:52,226 --> 00:24:53,998 And those are the things we really need to evaluate. 311 00:24:53,998 --> 00:24:54,808 And then... 312 00:24:55,465 --> 00:25:03,131 I think as well comparing the vendor AI to a well-promoted LLM, you know, and checking if the product actually adds value. 313 00:25:03,131 --> 00:25:12,558 Because again, these strong benchmarks should really assess like the accuracy, the consistency and the speed and the ability to handle these messy complex documents beyond 314 00:25:12,558 --> 00:25:15,481 just saying, can you get a simple question right? 315 00:25:15,481 --> 00:25:18,993 asking who are the parties, you know, what is the duration of the contracts? 316 00:25:18,993 --> 00:25:22,666 Like we can all do that, know, chat GPT can do that incredibly well. 317 00:25:22,666 --> 00:25:24,788 How well can you actually handle the real world legal? 318 00:25:24,788 --> 00:25:25,783 questions. 319 00:25:26,092 --> 00:25:26,642 Yeah. 320 00:25:26,642 --> 00:25:37,289 What about like identifying low risk, high value use cases where AI can like immediately benefit legal teams? 321 00:25:37,289 --> 00:25:40,020 Like what's a good, what's a good strategy for that? 322 00:25:40,756 --> 00:25:41,476 Yeah. 323 00:25:41,476 --> 00:25:51,316 So I think, mean, well, the easiest immediate high value use cases that I personally see is trying to automate the non-millipede stuff, like the time consuming stuff and, you 324 00:25:51,316 --> 00:25:58,136 know, the stuff that, you know, when people were watching suits back in the day, they weren't looking at going, oh, I wish I was going to do that task one day. 325 00:25:58,136 --> 00:26:05,216 I hope I could be buried in paperwork or hope I'm just sat there triaging requests that are coming in like, no, they were going, I want to do the call cases. 326 00:26:05,216 --> 00:26:06,756 I want to do all the fun stuff. 327 00:26:06,756 --> 00:26:10,656 And so if, you know, if something as simple as like triaging incoming emails. 328 00:26:10,684 --> 00:26:14,566 shift that unbillable time into more productive revenue generating work. 329 00:26:14,566 --> 00:26:25,693 again, if you're receiving hundreds of clients or emails daily, some urgent, some routine, some completely irrelevant, AI can categorize and prioritize those on urgency, on the 330 00:26:25,693 --> 00:26:27,274 client, on the subject matter. 331 00:26:27,274 --> 00:26:35,368 And again, if you've got a system like you were talking about before where you have the finance context, you can actually prioritize on how much are we billing this client? 332 00:26:35,368 --> 00:26:37,073 How often are we billing this client? 333 00:26:37,073 --> 00:26:38,994 and do urgency based on that as well. 334 00:26:38,994 --> 00:26:47,801 You know, can extract the key deadlines and action items from the email threads and then the intelligent routing, which is what I really love about it is going, well, who's the 335 00:26:47,801 --> 00:26:52,134 right lawyer or who's the right department to be doing this based on the content that's coming in? 336 00:26:52,134 --> 00:26:54,556 And again, like summarizing length of the email chains. 337 00:26:54,556 --> 00:26:56,383 Like I was using that recently. 338 00:26:56,383 --> 00:27:04,146 I just got off an extra couple of weeks of facility leave and I was taking these big email chains and getting AI to summarize, you know, what was the email chain? 339 00:27:04,146 --> 00:27:08,577 Am I still needed in this email chain, drafting an email to confirm if I'm not needed or not? 340 00:27:08,577 --> 00:27:16,519 And it was amazing to think of like, compared to what I was doing a year ago, where I was just working up the chain from the bottom of the email, getting to the second to last 341 00:27:16,519 --> 00:27:21,160 email and realizing I'm not needed anymore and wasting 15 minutes of my day. 342 00:27:21,160 --> 00:27:29,732 And obviously like, you know, these sort of seem like small efficiency gains, but you know, shaping minutes off every email review, compounds over and over, much like your 343 00:27:29,732 --> 00:27:33,283 pension does, into hundreds of hours saved each week. 344 00:27:33,315 --> 00:27:42,860 And if that can be redirected to billable work and high strategic value tasks, then it's a really good way to be doing it because like I've always said, AI should be replacing 345 00:27:42,860 --> 00:27:48,603 Claudius, but hopefully it should quite quickly eliminate those repetitive administrative tasks. 346 00:27:48,603 --> 00:27:52,085 allowing the legal professionals to work on what actually moves the needle for the clients. 347 00:27:52,085 --> 00:27:54,977 know, the clients are going, that's really good email tree. 348 00:27:54,977 --> 00:27:56,507 I shouldn't have done that. 349 00:27:56,648 --> 00:27:59,429 No one's saying that, but actually going, that was great. 350 00:27:59,429 --> 00:28:00,522 doctor really drafted it for me. 351 00:28:00,522 --> 00:28:01,142 That was really quick. 352 00:28:01,142 --> 00:28:03,079 How did you manage to spend so much time on that? 353 00:28:03,079 --> 00:28:05,739 Well, I wasn't triaging emails all day. 354 00:28:06,594 --> 00:28:07,775 Yeah, good, good point. 355 00:28:07,775 --> 00:28:21,402 I've been, I've been beating that drum for a while in terms of leveraging AI as a starting point, leveraging AI for business of law use cases is a good risk reward balance, right? 356 00:28:21,402 --> 00:28:25,104 Because it allows, it's a, it's an incremental step forward. 357 00:28:25,185 --> 00:28:35,050 It's a, there's still an opportunity cost that you're eliminating in terms of administrative time and you don't have to 358 00:28:35,084 --> 00:28:46,302 worry about client data in most scenarios and all of the OCGs that I saw a really interesting survey from Legal Value Network. 359 00:28:46,302 --> 00:28:51,796 do a legal project management survey. 360 00:28:51,796 --> 00:29:02,798 And one of the questions was, what percentage of your clients prohibit or discourage the use of GEN.ai on the 361 00:29:02,798 --> 00:29:03,998 resolution of your matters. 362 00:29:03,998 --> 00:29:17,458 And the answer was 42 % either prohibit or discourage the use of AI, which I thought was shockingly high because I've seen other numbers where clients, in fact, I think it was, 363 00:29:17,458 --> 00:29:22,238 um, it was from the, uh, blickstein groups. 364 00:29:22,238 --> 00:29:31,578 do a law department organization, LDO survey, uh, every year and almost 60 % of clients said, 365 00:29:32,110 --> 00:29:38,710 that law firms don't use technology enough to reduce costs. 366 00:29:38,710 --> 00:29:42,450 And it's like, all right, those are two conflicting messages. 367 00:29:42,450 --> 00:29:52,210 If you've got clients who want you to use tech to eliminate the costs, but then are discouraging or prohibiting you from using the tech to reduce the cost, we have a little 368 00:29:52,210 --> 00:29:55,610 bit of a tension there, which I don't really know what to make. 369 00:29:55,610 --> 00:29:55,890 I don't know. 370 00:29:55,890 --> 00:29:56,978 You got any insight on that? 371 00:29:56,978 --> 00:30:07,058 I mean, I'm very much getting flashbacks of when there was a little chores around the client turns around the The morphers cloud is like, well, I don't want to use the cloud. 372 00:30:07,098 --> 00:30:08,338 What about my security? 373 00:30:08,338 --> 00:30:15,338 I was like, I'm not sure where you think your emails go between you and us, but it's going to be the cloud at some point. 374 00:30:15,338 --> 00:30:20,838 So, and again, I think it's positioning, I it's positioning the tech in the correct format. 375 00:30:20,838 --> 00:30:22,963 Cause I think there was so much like foot. 376 00:30:22,963 --> 00:30:27,345 like the fear, uncertainty and doubt around the cloud and around AI. 377 00:30:27,345 --> 00:30:36,228 And it's trying to position itself the cloud going, know, we're not sitting here and going, the AI is doing this entire job unassisted and no one's looking at it. 378 00:30:36,228 --> 00:30:46,472 What we're trying to do here is we're trying to reduce the pain points that we're all facing in law, but currently he's taking up our time doing your work, getting us to do the 379 00:30:46,472 --> 00:30:49,093 interesting stuff that you actually value here. 380 00:30:49,113 --> 00:30:52,854 I was chatting to a friend who works at Adelshaw's and they were talking. 381 00:30:52,854 --> 00:31:02,357 about projects they've done in the past and something that's similar to WeHoTel, which is around, you know, when you look at a huge data room, you can use generative AI to 382 00:31:02,357 --> 00:31:08,818 basically take all the documents, look at them, summarize them, check is the title actually correct? 383 00:31:08,878 --> 00:31:15,020 You know, like, is this actually an NDA or is this actually, you know, an employment contract? 384 00:31:15,100 --> 00:31:22,442 And reviewing the contents of the document and going actually, let's probably update this title, like from NDA to employment contract. 385 00:31:22,961 --> 00:31:26,301 And like, that's a huge task for someone to be doing. 386 00:31:26,301 --> 00:31:33,021 You know, like you give up to power legal and that's probably like three weeks worth of work to get through like 10,000 documents or so. 387 00:31:33,541 --> 00:31:39,101 But using AI to do that seems like a really easy, again, low hanging fruit piece to try and do. 388 00:31:39,101 --> 00:31:41,541 It's high impact, low risk. 389 00:31:41,781 --> 00:31:44,521 And you can also do document summarization at the same time. 390 00:31:44,521 --> 00:31:48,881 So can go, here's your document, we've fixed the title, we've fixed the categorization of it. 391 00:31:48,881 --> 00:31:50,381 Here's a summary of the document. 392 00:31:50,381 --> 00:31:50,881 Cool. 393 00:31:50,881 --> 00:31:52,041 Off we go. 394 00:31:52,221 --> 00:32:00,348 I think it's all about the framing of AI because I think AI landed in a way that no other legal tech ever has. 395 00:32:00,969 --> 00:32:03,471 Document automation was never on the BBC News. 396 00:32:03,471 --> 00:32:10,316 There was never an emergency worldwide conference hosted by the primaries of the UK around document automation. 397 00:32:10,537 --> 00:32:13,279 People had a lot more visibility into it. 398 00:32:13,279 --> 00:32:15,921 And again, it's on their devices so quickly. 399 00:32:15,921 --> 00:32:20,685 Alexa are doing their Alexa Plus in the coming months or so in the US where... 400 00:32:20,785 --> 00:32:24,865 Genitive AI will be in everyone's homes all of sudden. 401 00:32:25,665 --> 00:32:32,725 And I think that just then gives people expectations, but also gives them that issue where they go, well, hold on, what is this actually doing? 402 00:32:32,725 --> 00:32:36,905 Because again, it is still a black box for a lot of internal testing purposes. 403 00:32:37,545 --> 00:32:44,665 then researchers are really trying to delve into, how does it make the decisions that it makes and why does it anchor onto certain concepts that it does? 404 00:32:44,665 --> 00:32:48,081 But it's quite hard to do that still, so it is. 405 00:32:48,081 --> 00:32:53,341 In the same way that it's quite hard sometimes for a human to explain what they're doing and how they've approached a particular task. 406 00:32:53,541 --> 00:33:00,821 So I think that's main thing is how we frame AI and how we frame where we're using AI to really get people comfortable with it. 407 00:33:00,821 --> 00:33:08,141 Because like you say, if we're not using like, you know, we've exhausted a lot of the tech that was bringing benefits to people. 408 00:33:08,141 --> 00:33:12,541 And you know, we have teams, have emails, we're not doing postal stuff anymore. 409 00:33:12,541 --> 00:33:13,096 So. 410 00:33:13,296 --> 00:33:16,016 AI is the next big revolutionary for me. 411 00:33:16,016 --> 00:33:16,996 It's huge. 412 00:33:16,996 --> 00:33:21,196 It feels the same way that the internet did back when I was a kid. 413 00:33:21,196 --> 00:33:22,316 was really excited. 414 00:33:22,316 --> 00:33:28,976 When I was a kid, the internet came, was like, oh my goodness, I can read, can talk to people, I can do this, I can do that, I can go on forums. 415 00:33:29,036 --> 00:33:31,996 And AI sort of had that same sort of excitement for me. 416 00:33:31,996 --> 00:33:33,856 Like, this is incredible. 417 00:33:34,336 --> 00:33:37,256 It's able to do things that we've not seen before. 418 00:33:37,576 --> 00:33:42,864 mean, talking beyond sort of tech, the vision capabilities, you could basically allow. 419 00:33:42,864 --> 00:33:46,204 blind people to read newspapers and give them context. 420 00:33:46,524 --> 00:33:49,144 Even of the stuff which is very visually heavy. 421 00:33:49,144 --> 00:33:56,284 I remember I was at my parents' house and there was a wine review and there was a table with some wines, some prizes, some texts underneath them. 422 00:33:56,284 --> 00:34:01,844 And I took a picture and asked ChatGVT to structure this in a way that I could then read back out. 423 00:34:01,844 --> 00:34:07,684 And it took these sort of inferences, like there was red text inferred ahead of it, but it wasn't like a structured table. 424 00:34:07,684 --> 00:34:11,384 It was really sort of quite an informal way of presenting the data. 425 00:34:11,736 --> 00:34:19,146 And that's an amazing accessibility piece that generative AI has enabled that wasn't available before. 426 00:34:19,146 --> 00:34:26,435 again, I think it's sort of showing those real benefits to people and showing where we can start to use it tactfully in our workflows. 427 00:34:26,988 --> 00:34:33,861 Yeah, there's all sorts of interesting insights that previously would never be discovered. 428 00:34:34,441 --> 00:34:42,215 I went to TLTF this year in Miami and in the lobby of the Ritz Carlton, they had a gingerbread house, you know, it was December. 429 00:34:42,215 --> 00:34:46,327 They had a gingerbread house, the size of like a Volkswagen Beetle. 430 00:34:46,327 --> 00:34:48,568 And my wife was like, that's so cool. 431 00:34:48,568 --> 00:34:53,930 You know, and they had the list of ingredients that was required to make it. 432 00:34:54,178 --> 00:34:55,699 So I was like, you know what'd be interesting? 433 00:34:55,699 --> 00:34:59,441 I was like, I wonder how many people that this house could feed. 434 00:34:59,501 --> 00:35:08,876 And so I took a picture of the ingredient list and asked chat GPT, how many cookies could you make with the same number of ingredients? 435 00:35:08,876 --> 00:35:10,047 And the answer was like 26,000. 436 00:35:10,047 --> 00:35:13,988 Like there were 26,000 cookies. 437 00:35:14,089 --> 00:35:15,820 It completely changed my wife's perspective. 438 00:35:15,820 --> 00:35:19,252 She's like, she went from, that's amazing to that's horrible. 439 00:35:19,252 --> 00:35:20,142 What a waste. 440 00:35:20,142 --> 00:35:23,842 Like we got people starving in the streets and we have this 441 00:35:23,842 --> 00:35:26,624 food here that we're using as decoration. 442 00:35:26,825 --> 00:35:38,644 And you know, I've, uh, I was talking to another guy down in Miami, the same conference, and he was, uh, his basement flooded and he was retired. 443 00:35:38,644 --> 00:35:50,654 had to retile his basement and he had all these boxes of tiles and he took a picture and asked chat GPT how much in, in, you know, square footage. 444 00:35:50,654 --> 00:35:53,846 And he also took a picture of the floor plan. 445 00:35:54,070 --> 00:36:00,292 And you know, how much, surplus or, you know, short was he with the tie and it gave him the answer. 446 00:36:00,292 --> 00:36:10,645 And it's just like, you know, again, that's the, those aren't business scenarios, but just w but just by taking a picture, can get, and it re you know, it read the box and the size, 447 00:36:10,645 --> 00:36:13,055 the dimensions of the tiles and the number of tiles in the boxes. 448 00:36:13,055 --> 00:36:15,556 And he had like 50 boxes. 449 00:36:15,676 --> 00:36:17,747 So it's, uh, it's incredible. 450 00:36:17,747 --> 00:36:22,978 The things that you can gain insight on that previously would have been out of reach. 451 00:36:23,630 --> 00:36:32,350 Yeah, and I think it's even when you go back to the business world, when I was looking at the Alexa memo recently and they saying like, the plan is for them to be a bit more 452 00:36:32,350 --> 00:36:33,850 proactive going forward. 453 00:36:33,850 --> 00:36:37,550 So you could be like, well, have a look at my calendar and see what's coming up. 454 00:36:37,750 --> 00:36:44,310 you know, you know, I've been having conversations, have I said, oh, let's catch up when I'm in London next, you know, so when am I in London next? 455 00:36:44,310 --> 00:36:49,935 Can we check their calendar as well and do all those kinds of things, which takes a lot of work. 456 00:36:49,935 --> 00:36:56,195 It tends to be why people don't end up seeing each other as much because I forgot to do that extra stuff, I didn't have the time. 457 00:36:56,195 --> 00:37:03,995 Well, having that sort of AI system being more proactive in a privacy preserving way just seems like such a cool thing to have. 458 00:37:03,995 --> 00:37:06,795 It could be like, know, do I need to buy any birthday presents? 459 00:37:07,215 --> 00:37:11,755 well, I can see that you're going to be going to such a place soon, the person you know who lives there. 460 00:37:11,755 --> 00:37:14,455 So if you buy a present now, you'll be able to take it with you if you go there. 461 00:37:14,455 --> 00:37:17,835 But if you're not going there, then maybe you need to post something now. 462 00:37:17,835 --> 00:37:18,690 It's like... 463 00:37:18,690 --> 00:37:27,442 those really nice little features, which again, AI assistant, well, know, Alexa just wasn't have had like, you know, four years ago until LMS arrived. 464 00:37:27,442 --> 00:37:29,573 think that tech was just so out of reach for them. 465 00:37:29,573 --> 00:37:31,944 I think bringing it in now is such a thing. 466 00:37:31,944 --> 00:37:39,736 And again, it starts to sort of almost like tip the balance maybe in the future as clients are going, well, actually AI is really actually quite useful for me at home. 467 00:37:39,736 --> 00:37:42,617 Like I'm finding so much utility into it. 468 00:37:42,617 --> 00:37:47,538 So actually maybe I'm not going to be so concerned about the legal firms usage. 469 00:37:48,280 --> 00:37:48,991 Yeah. 470 00:37:48,991 --> 00:38:00,401 The, uh, you know, and right now I feel like there's such a, an explosion of exploration that's happening with AI in, in the legal world. 471 00:38:00,401 --> 00:38:12,623 I have a friend of mine who he had a, I don't know the details of his exit, but I, I know enough to know he did pretty well and he started another company that is AI focused and 472 00:38:12,623 --> 00:38:13,750 he's older than me. 473 00:38:13,750 --> 00:38:14,218 I, 474 00:38:14,218 --> 00:38:16,010 first thing I asked him was like, what the hell are you doing, man? 475 00:38:16,010 --> 00:38:16,890 Go enjoy your money. 476 00:38:16,890 --> 00:38:17,961 Why are you doing this again? 477 00:38:17,961 --> 00:38:19,742 Why you putting yourself through this? 478 00:38:19,903 --> 00:38:29,241 But anyway, he created a solution that is AI focused and it could be used outside of legal. 479 00:38:29,241 --> 00:38:31,715 He just happens to have his Rolodex in legal. 480 00:38:31,715 --> 00:38:33,374 he started there. 481 00:38:33,494 --> 00:38:43,232 Well, he said that even when he has executive sponsorship, like the managing partner of the firm who wants to evaluate his platform, 482 00:38:43,266 --> 00:38:59,058 He is now getting put into a queue and a prioritization committee is managing when the firm will take a look because they have so many POCs and pilots in AI right now that 483 00:38:59,058 --> 00:39:00,419 they're being overwhelmed. 484 00:39:00,419 --> 00:39:10,506 like, I don't know, do you have any thoughts on how firms balance like the exploration piece with sustainable solutions in the longterm? 485 00:39:11,021 --> 00:39:21,601 Yeah, it's a really interesting one because I think that the whole sort of big shifts I feel like in AI sort of allow legal teams to really prototype ideas quickly without 486 00:39:21,601 --> 00:39:27,201 needing like dev team resources like myself, like, you know, do you need sentiment analysis on case data? 487 00:39:27,201 --> 00:39:29,081 Cool, AI will do that in a second. 488 00:39:29,121 --> 00:39:36,801 You want to test contract clause extraction and detection, know, AI can validate if that's going be useful or not before you need a full automation. 489 00:39:36,881 --> 00:39:38,437 But yeah, I feel like 490 00:39:38,862 --> 00:39:50,062 A lot of the problem is there's so many AI vendors at the moment, they've all got potentially really good tools, built predominantly on lots of foundation models, but it's 491 00:39:50,062 --> 00:40:00,742 that issue that we have around benchmarking is a lot of firms aren't necessarily in a position to be able to say, how do we test this effectively before we invest in this? 492 00:40:01,382 --> 00:40:07,149 And like you saying about your friend there, some vendors who are really focusing on the legal market could 493 00:40:07,149 --> 00:40:09,229 quite easily pivot away from legal. 494 00:40:09,669 --> 00:40:13,049 you could, know, know, selling to live law firms is slow. 495 00:40:13,209 --> 00:40:21,909 you know, procurement takes a long time, there's a lot of risk and regulatory requirements and you know, if the adoption is slow, then AI startups are probably just going to shift 496 00:40:21,909 --> 00:40:26,009 their focus entirely, like finance or compliance or healthcare. 497 00:40:26,369 --> 00:40:34,549 And I think maybe sometimes law firms are looking at it going, well, because we're slow, the vendors are pivoting and are they actually going to support our legal workflows going 498 00:40:34,549 --> 00:40:35,569 forward? 499 00:40:35,909 --> 00:40:37,005 Or, you know, 500 00:40:37,005 --> 00:40:38,705 Would there be an acquisition? 501 00:40:38,985 --> 00:40:46,985 know, cause you know, we saw a Thomson Reuters picking up so many companies very low as a case text for $650 million or so. 502 00:40:47,045 --> 00:40:55,305 And if you think, well, I was a case, you know, I wasn't, if I was a case test customer, like it's now just part of the Thomson Reuters Code Council product to say, do I want 503 00:40:55,305 --> 00:40:55,705 that? 504 00:40:55,705 --> 00:40:56,745 I didn't really want that. 505 00:40:56,745 --> 00:40:58,505 I just wanted what case texts were offering. 506 00:40:58,505 --> 00:41:02,949 it's, you know, if that tool gets rolled into a larger platform, you might not want it. 507 00:41:03,053 --> 00:41:09,853 Pricing can change what was previously a nice flexible standalone tool becomes a big enterprise licensing bundle. 508 00:41:10,453 --> 00:41:15,513 you know, it's really interesting that piece around like AI solutions appearing. 509 00:41:15,513 --> 00:41:17,953 are going, oh, cool, it's really got benefit in legal. 510 00:41:17,953 --> 00:41:23,013 But actually a lot of the benefit in legal with the AI stuff is applicable to so many industries. 511 00:41:23,013 --> 00:41:28,053 Like a lot of industries, a lot of them are just dealing with documents in the same way that legalists. 512 00:41:28,053 --> 00:41:29,613 We need get information out of documents. 513 00:41:29,613 --> 00:41:31,190 We need to categorize them. 514 00:41:31,190 --> 00:41:32,361 We need to get our insights. 515 00:41:32,361 --> 00:41:33,592 We need to do reporting. 516 00:41:33,592 --> 00:41:35,434 We need to do email summarisation. 517 00:41:35,434 --> 00:41:39,027 So legal is a really high value market to AMA. 518 00:41:39,027 --> 00:41:43,561 And there's a lot of lawyers who want to make use of this and get more efficiency out of their work. 519 00:41:43,561 --> 00:41:45,984 But there's so many other industries who want the same. 520 00:41:45,984 --> 00:41:53,540 And if they're willing to adopt quicker, then legal may find itself stuck without as many tools as it thought it may well have had originally. 521 00:41:53,752 --> 00:41:55,303 Well, and that's exactly what he's doing. 522 00:41:55,303 --> 00:42:01,909 He's pivoting away from legal because he's stuck in the mud at all these firms. 523 00:42:01,909 --> 00:42:02,760 You know, it's interesting. 524 00:42:02,760 --> 00:42:06,413 know, the pendulum, I feel like it's swinging back in the other direction. 525 00:42:06,413 --> 00:42:08,775 I've actually, it's incredible. 526 00:42:08,775 --> 00:42:18,043 The number of emails from VCs that I get, you know, wanting to get to know, you know, they see our growth on LinkedIn. 527 00:42:18,043 --> 00:42:21,856 I'm assuming, I don't know how else we would hit the radar. 528 00:42:21,996 --> 00:42:24,889 I could spend 40 hours a week just meeting with VCs. 529 00:42:24,889 --> 00:42:26,560 So I don't, I don't meet with any of them. 530 00:42:26,560 --> 00:42:30,373 We have one investor TLTF and like, that's all. 531 00:42:30,373 --> 00:42:31,894 It just doesn't make sense. 532 00:42:31,894 --> 00:42:45,546 But I have had a few casual conversations and what I'm hearing now is from also investors who feel maybe a little overexposed on with their portfolio on AI. 533 00:42:45,546 --> 00:42:48,648 You know, it went so hard, so fast. 534 00:42:48,648 --> 00:42:51,040 And now they're looking for companies that 535 00:42:51,040 --> 00:43:02,216 aren't necessarily gen AI exclusive and, who have, can demonstrate growth and good product market fit. 536 00:43:02,216 --> 00:43:13,122 So now we're seeing an interesting, again, the pendulum is swinging back a little bit where you got companies that are pivoting to, to markets where they can actually get some 537 00:43:13,122 --> 00:43:18,005 traction because there's such a glut of tools, point solutions out there. 538 00:43:18,005 --> 00:43:19,566 You've got investors. 539 00:43:19,566 --> 00:43:26,289 who are now looking for firms that AI isn't the central story to their narrative. 540 00:43:26,289 --> 00:43:28,520 So, you know, it's going to be interesting. 541 00:43:28,520 --> 00:43:38,724 And then combine that with, you know, there are some companies out there with some really big valuations where I don't think their revenue is nearly as stable. 542 00:43:38,724 --> 00:43:42,155 Yeah, they have great growth, but how much churn are they going to have? 543 00:43:42,155 --> 00:43:47,087 Because I hear stories off the record, like, yeah, this isn't actually working that great. 544 00:43:47,087 --> 00:43:48,278 And it's really expensive. 545 00:43:48,278 --> 00:43:49,068 And 546 00:43:49,198 --> 00:43:49,838 I don't know. 547 00:43:49,838 --> 00:43:54,566 Do you, do you feel like the pendulum is swinging back a smidge or, or no. 548 00:43:54,699 --> 00:44:04,839 Yeah, it's a funny one because I always think back to when I see all these huge valuations, know, prepared to what they're earning, I always sort look at Uber and how 549 00:44:04,839 --> 00:44:13,739 Uber integrated itself and like, you know, the taxes were so cheap back in the day, know, Uber was backed by so much VC money and then the VC went, well, hold on, where's my return 550 00:44:13,739 --> 00:44:14,459 here? 551 00:44:14,459 --> 00:44:18,419 And now Uber's are suddenly like, seemingly like 10 times the cost where I live. 552 00:44:18,419 --> 00:44:23,465 I'm like, oh, actually it's not really the great value I once thought it was because, know, VC have gone, I want my money. 553 00:44:23,465 --> 00:44:32,909 now kind of thing and it'd be interesting to see like you know will they be able to struggle, will they struggle to sort of maintain that I think especially if you're looking 554 00:44:32,909 --> 00:44:41,512 at tools where they're really targeting sort of prestige markets you know there's only so many big law firms around like to sort of have that and you know get them out of seats 555 00:44:41,512 --> 00:44:50,920 that you need whereas actually the middle market and the sort of smaller firms you know there's thousands or millions of those across the world so I 556 00:44:50,920 --> 00:44:52,971 Will they start targeting those instead? 557 00:44:52,971 --> 00:44:59,504 And obviously that then increases your sort of how much support you may have to do and then further eroding how much profit you can make. 558 00:44:59,504 --> 00:45:05,846 So yeah, it's, it's interesting to see how these sort of like really big legal, legal AI tools. 559 00:45:05,846 --> 00:45:10,678 It's like, I'm interested to see how they make their money back based on these valuations. 560 00:45:10,872 --> 00:45:22,889 Yeah, because investors in the VC world, they have to hit home runs one out of 10 times where the numbers just don't work. 561 00:45:22,889 --> 00:45:32,664 And those home runs, I if you look at Harvey at a $3 billion valuation, they're really too big to be acquired at this point, right? 562 00:45:32,664 --> 00:45:37,667 So it's a go public or bust scenario from an investor perspective. 563 00:45:37,667 --> 00:45:39,874 And is there enough of a TAM? 564 00:45:39,874 --> 00:45:42,536 you know, a total addressable market for that to happen. 565 00:45:42,536 --> 00:45:47,554 I don't, I'm not pretending to have the answer and they've got some incredible investors in their cap table. 566 00:45:47,554 --> 00:45:51,141 I mean, Google ventures, open AI, a 16 Z. 567 00:45:51,141 --> 00:45:59,656 I'm assuming they've done the math writing these big checks, but from an outsider, man, I'm having a hard time connecting the dots on how is this going to work? 568 00:46:00,490 --> 00:46:03,670 Yeah, it's crazy when talk about it, right? 569 00:46:03,670 --> 00:46:06,730 was like, well, you know, surely there can't be that much demand for it. 570 00:46:06,730 --> 00:46:10,450 But, know, it seems like there must be for the valuations that they're getting. 571 00:46:10,450 --> 00:46:14,070 Because like you say, they've got incredibly smart people working there. 572 00:46:14,070 --> 00:46:19,930 know, we've got, know, PhD students, PhD graduates, guess is what you call them. 573 00:46:19,930 --> 00:46:28,410 They have really incredible sort of legal technology people there beyond just like, oh, well, here's a thin wrapper around, you know, chat GBT, which a lot of the tools were very 574 00:46:28,410 --> 00:46:28,809 early on. 575 00:46:28,809 --> 00:46:33,549 that's all you need to do and you really need to differentiate you know now. 576 00:46:33,709 --> 00:46:45,409 So again it'll be interesting to see how that evolves over time and again like say with your friend like do they need to pivot into other markets going forward because again the 577 00:46:45,409 --> 00:46:55,571 issues that legal face are issues that other markets face and again can they break into like GCs again you only know maybe one or two licenses in every GC but there are 578 00:46:55,571 --> 00:46:57,241 millions of firms around the world. 579 00:46:57,241 --> 00:47:00,069 We've all got a couple of lawyers working there so yeah. 580 00:47:01,080 --> 00:47:02,280 Yeah, for sure. 581 00:47:02,280 --> 00:47:14,764 think that's definitely, I don't know how it's kind of a little bit like if you're selling to law firms, but you're also selling to GCs and making them more self-sufficient, you're 582 00:47:14,764 --> 00:47:17,585 taking away from law firm revenues. 583 00:47:17,585 --> 00:47:22,036 So I don't know how you straddle that fence, but they seem to be doing well. 584 00:47:23,256 --> 00:47:24,237 I wish them the best. 585 00:47:24,237 --> 00:47:29,272 want to see, I want to see a success, especially from, you know, which is 586 00:47:29,272 --> 00:47:31,303 what is arguably the leader in the space. 587 00:47:31,303 --> 00:47:41,183 Certainly they've got, you don't see, you know, I've been around the space for a long time, almost 20 years in legal tech and you don't see investors like that in legal tech 588 00:47:41,183 --> 00:47:42,975 cap tables, right? 589 00:47:42,975 --> 00:47:43,935 Like ever. 590 00:47:43,935 --> 00:47:45,617 I've never seen that. 591 00:47:45,617 --> 00:47:49,020 So I, I want, you know, the investment is good. 592 00:47:49,020 --> 00:47:50,601 I want the investment to keep flowing. 593 00:47:50,601 --> 00:47:51,692 So I wish them the best. 594 00:47:51,692 --> 00:47:56,136 It's just, uh, yeah, I, they know some things I don't. 595 00:47:56,873 --> 00:48:06,293 Yeah, mean, again, they must have incredible, incredibly persuasive talents, which again, I think is where maybe a lot of other legal tech has failed in the past because maybe 596 00:48:06,293 --> 00:48:15,013 you've had start-up series, know, it's a, know, probably maybe a developer and a lawyer and they're going, you know, we really focused on the tool, but actually, like you say, 597 00:48:15,013 --> 00:48:20,168 for a lot of things you see, it's not how amazing the tool is initially, it's how well you can market it how, you know. 598 00:48:20,168 --> 00:48:27,828 how well your contacts are in getting into these big firms and being able to talk to people and convincing them this is where they should be going going forward. 599 00:48:28,128 --> 00:48:38,068 Especially when you think of how slow law firms work and how slowly they move in terms of adoption of tools, being able to convince them to adopt your tool and adopt it quickly is 600 00:48:38,068 --> 00:48:39,288 an incredible talent. 601 00:48:39,288 --> 00:48:44,288 And I wish I had as much of that trying to get some internal tool adoption at times. 602 00:48:44,354 --> 00:48:46,115 Yeah, absolutely. 603 00:48:46,115 --> 00:48:47,716 Well, we're, we're about out of time. 604 00:48:47,716 --> 00:48:49,097 This has been a great conversation. 605 00:48:49,097 --> 00:48:55,540 I know we, uh, we, kinda went off script a little bit with the KPMG stuff, but it's, it's so timely. 606 00:48:55,540 --> 00:49:04,045 I'm actually recording a podcast tomorrow with Debbie Foster from affinity consulting, and we're going to talk about the big fours push. 607 00:49:04,045 --> 00:49:06,156 Um, yeah, I'm a, I'm not a legal guy. 608 00:49:06,156 --> 00:49:07,116 I don't have a JD. 609 00:49:07,116 --> 00:49:12,369 I've got an MBA and I'm, I consider myself kind of a legal market analyst. 610 00:49:12,369 --> 00:49:14,470 Um, and I, 611 00:49:15,195 --> 00:49:18,017 All of these competitive dynamics really interest me. 612 00:49:18,017 --> 00:49:25,332 So yeah, it's going to be an interesting next couple of years to see, to see how this plays out. 613 00:49:25,596 --> 00:49:31,215 Yeah, I'm going to have a very busy time building out more tooling to support our US firm, so it's going to be great fun. 614 00:49:31,382 --> 00:49:33,686 Yeah, yeah, you got good job security. 615 00:49:33,686 --> 00:49:39,574 All right, well, I really appreciate you joining this afternoon and let's definitely stay in touch. 616 00:49:39,912 --> 00:49:42,082 Yeah, much appreciated, speak to you soon. 617 00:49:42,082 --> 00:49:43,785 All right, thanks, Ryan. -->