A structural or MEP firm that brings in a drafting agent is handing a machine the first pass at pages that eventually carry a seal. The principal who approves that decision should have a short list of questions that sort serious vendors from vague ones. These eight cover the ground, with the answer worth listening for under each.
Who reviews and seals every page?
The answer should name a role, not a process: the engineer of record, or a designated reviewer working for that engineer. If the vendor says the agent's output is "validated," ask by whom. In a licensed practice, a person with a license reads the page.
Can the agent show where each statement came from?
A plan check reply that says the beam was checked for the load in the criteria memo should point to the memo, the revision and the page. The answer you want is that every statement comes with a quoted source from your own files, and that a statement without a source is marked as such.
What happens when the sources disagree?
Calculations and drawings drift apart during a project. A good agent notes the conflict and stops, leaving the choice to the engineer. A poor one picks a version. Ask for an example with two conflicting revisions and watch what it does.
What does it never do?
The vendor should state the boundaries without being pushed: it does not run analysis, does not change a design value, does not send anything to a reviewing agency and does not mark a comment resolved. A long list of never statements is a good sign.
Where does our project data go?
The firm's calculations, client drawings and site reports are confidential. Ask where files are stored, who at the vendor can see them, whether they are used to train any model and how they are deleted at the end of an engagement. Expect specifics, including the contract language that backs them up.
How do we see what it did?
Every draft needs a record: the prompt, the files consulted, the version of the instructions and the time. If a question comes back from a building department a year later, the firm should be able to reconstruct how a reply was produced.
How does the instruction set change over time?
Your firm's standards will change, and so will the codes it works under. Ask who edits the agent's instructions, how an edit is reviewed and how an older reply is tied to the instructions in force when it was written.
What do we do if it is wrong?
The honest answer describes the review that catches the error, the correction that follows and the change to the instructions so the same error does not recur. A vendor who says errors will not happen has not worked in a design office.
Reading the answers together
Taken as a set, these questions test one thing: whether the vendor treats a licensed engineer's review as the center of the system or as a formality. An agent that quotes sources, stops at conflicts and keeps a record fits a practice where the seal means something. One that cannot answer in specifics is better left out of the office, however good its demonstration looks.
It also helps to ask the same questions of your own firm. Who would review an agent's page on a busy week? Which project folders are off limits? Answers that come slowly inside the office will come slowly in the vendor's contract as well, and it is better to find that out at the first meeting.