In early November Sam Altman announced some new models and developers’ products, taking OpenAI one step closer to becoming a true platform. One of the new capabilities announced for GPT-4 was the ability for users to create their own chatbots and agent-like experiences with a click.
We were excited to introduce Pelles GPT, a specialized tool exclusively designed for MEP subcontractors’ estimators and engineers. After reviewing its benefits (here and here) it is now time to discuss its limitations and disadvantages or in other words — why GPT isn’t good enough for the industry and why the industry should have dedicated AI tools.
We find GPT to be less than ideal for construction professionals to use in their daily workflow because of these top 5 limitations listed below.
To check GPT’s limitations we uploaded a bid package and asked it some questions. We used a bid package for mechanical works for the renovation of a ~4,000-square-foot restaurant in NY.
At first sight, this seems like a legitimate answer. However, this answer fabricates facts and demonstrates the lack of common sense of the GPT, as even a rookie estimator knows that these numbers don’t add up for a 4,000-square-foot project.
The GPT got the diffuser types right but the quantities wrong. To get the quantities right it needed industry-specific context; the number of diffusers isn’t shown on the schedule and has to be counted on the drawing (assuming the drawing is completed). It also needed high multimodal abilities to count drawings. Both of these necessities are lacking at the moment.
This is where those numbers came out of:
Additionally, this highlights the limits of working with the GPT while double-checking its sources.
Per the schedule, this project requires water-cooled DX systems of that model. Remember GPT doesn’t have real-time data? Instead of simply stating that it generates a long answer (with no answer).
Answering this question requires context, nuance, and common sense. To get it right, GPT needs not only to identify what is considered equipment but also to understand that the objective is to identify the major systems, not each fan or diffuser.
This is again, a very partial answer. It lacks common sense and context to know that the MAU and PCU are of importance. Unit designations (AC-1, AC-2, etc)are mistakenly taken as model numbers.
While OpenAI’s GPT already delivers impressive outcomes for MEP professionals, the real test lies in tailoring AI models to provide customized, high-value solutions specific to the MEP sector.
This endeavor presents an exciting yet unmastered frontier.