Can AI agents read construction drawings and build BIM models? I tested four

Full evaluation report (PDF, ~60 pages, in Chinese, with all methods, data and appendices): download

A lot of building-construction work is translation between drawings and models: reading a drawing set, modeling from it, adding detailing elements that the codes require, and tying a construction schedule to the model. The work is high-volume, rule-heavy and depends on experience. General-purpose LLMs can now write code and drive software inside their own agent environments, so it is reasonable to ask how far they get on this kind of BIM work, and which parts can be handed over today.

I tested four models: Opus 5.5, GPT-6 Astra, Kimi K3.1 Pro and DeepSeek V4.1 Flash. The tasks are real questions and drawings from a national BIM skills certification exam in China (construction track). This post describes how I ran the tests and what I found.

The short version

Working from five construction drawings, Opus 5.5 produced a first-draft model of a full basement level in about 30 minutes. The main elements are nearly all there and mostly in the right place, but wall thicknesses, column sizes and door/window tags still need a human check. GPT-6 Astra came out close to Opus. Kimi K3.1 Pro and DeepSeek V4.1 Flash left out a large share of the walls and are not yet usable for first drafts.

Figure 1. The basement level Opus 5.5 modeled from five construction drawings (IFC, roof slab hidden).

How I tested

Models and harnesses. Each model ran in its vendor’s own agent harness: Opus 5.5 in Claude Code, GPT-6 Astra in Codex, Kimi K3.1 Pro in Kimi Code, and DeepSeek V4.1 Flash in DeepSeek Harness. What I compare is therefore the whole system, model plus harness. Common settings: reasoning effort set to “high” everywhere, no web access, no sub-agents. Tools for viewing drawings (crop and zoom) and for working with each BIM format were provided as skills that explain how to use the tool and contain no hints about the answers.

Three ways to touch the BIM model. Every modeling task was run three times per model, once per toolchain:

• IFC: the model writes Python with IfcOpenShell and generates or edits an IFC file directly.
• BimDown: a simplified BIM model format designed for AI agents (AI-native). Element properties live in CSV tables and geometry in GeoJSON, with a command-line tool for validation and builds; the model reads and writes these text files directly.
• Revit: the model drives a live Revit session over MCP, either through modeling tools or by sending C# code to run inside Revit.

Four tasks. All material comes from two recent sittings of the certification exam. The reference answers come from the DWG and IFC source files that ship with the exam; wherever the exam’s IFC disagreed with the drawings, I corrected it to match the drawings, since that is what the models were reading.

Task	What the model has to do	Size
Drawing reading	7 question groups, 26 sub-questions: grid lines and spacings, counting fire-door tags, a pile-cap schedule, site-layout orientation, a ramp section, cross-checking architectural against structural plans, and the 3D make-up of a steel column joint	One session per question group
Modeling from drawings	Build an entire basement level from 5 drawings: walls, doors, windows, columns, shear walls, roof beams and slabs, pile caps, raft zones, sump pits, pour strips	About 520 elements
Secondary-structure detailing	Add constructional columns, ring beams, lintels and wall kerbs to every masonry wall on the level, following rules stated in the prompt	90 masonry walls, 45 openings
4D scheduling	Schedule the steel erection of a 13-storey tower and link each task to model elements	About 3,100 tower elements

A few terms need explaining for readers outside China:

• Pile caps are the concrete blocks that sit on top of groups of piles. Pour strips (also called post-cast strips) are narrow bands left out of a large concrete slab and cast weeks later, once most of the shrinkage and settlement has happened.
• Elevations are written in metres relative to the ground-floor finished level, so -4.400 means 4.4 m below ground floor (±0.000).
• Grid lines follow the Chinese convention: numbered lines in one direction, lettered lines in the other, and secondary grid lines written as fractions. For example, 1/4 is the first extra line after grid 4.
• The beam drawing uses China’s standard flat notation for reinforced concrete beams, in which each beam’s size and span count are written on the plan next to the beam rather than drawn in sections.
• Secondary structure is China’s seismic detailing for masonry infill walls. Small cast-in-place constructional columns are placed at wall ends, corners and junctions and at regular spacing. Ring beams (or bond beams) run along the wall, lintels span over openings, and a low concrete kerb runs along the base of the wall. Together they tie the infill into the frame.
• The 4D task comes with explicit rules: construction starts on 1 January 2026; the concrete core takes 3 days per floor; steel erection starts once the core reaches floor 5, with steel columns at 3 days per floor and steel beams at 5 days per floor; the roof steel must be lifted in sections; and the tower crane’s climbing schedule must keep the jib at least 8 m above the structure at all times.

The tasks don’t chain together. Each one starts from clean reference input, so a mistake in one task can’t carry into the next.

Scoring. The reading questions were graded blind against a rubric, 10 points per group, 70 in total. For modeling, every output, whatever the toolchain, is reduced to the same representation (plan footprint, elevation range, element type) and compared with the reference model. That gives two scores: detection (was the element built, and in the right place?) and accuracy (are wall thickness, column section, beam-top elevation, slab thickness, pile-cap size, door/window tags and so on within tolerance?). The detailing task is scored against the five rules in the prompt. For the 4D task I check ten scheduling rules and record which task each element was assigned to.

I used one pre-designated run per combination, with no cherry-picking. One rule mimics real use: if a result had a problem a user would spot immediately without an answer key (a whole element category missing, everything shifted, no deliverable), the run could be retried once, keeping the better result and counting the time and cost of both attempts. Only Kimi triggered this, twice.

Results

Drawing reading: all four can read a construction drawing

Opus 5.5 and GPT-6 Astra answered all seven question groups correctly. Kimi K3.1 Pro scored 64.75 out of 70 and DeepSeek V4.1 Flash 66.5. Most of the lost points came from the group that requires overlaying the architectural plan on the structural plan: given a room, state the elevation and thickness of the slab above it and list the beams around it. Kimi and DeepSeek misread the slab level or thickness in some areas.

This task did not separate the models much, and is best read as a baseline check. Being able to crop and zoom the drawing inside the agent loop helps a lot. With tools available, reading text, dimensions and symbols off a construction drawing is no longer a real obstacle for frontier models.

Figure 2. Total reading score out of 70. Darker bars: the drawing given as an image. Lighter bars: the original CAD file given instead.

Modeling from drawings: where the models differ most

This was the main task, and the gap between models was largest here. Across the three toolchains, Opus 5.5 scored 0.92 to 0.96 on detection and 0.86 to 0.91 on accuracy, and GPT-6 Astra 0.86 to 0.91 and 0.82 to 0.86. Kimi K3.1 Pro and DeepSeek V4.1 Flash were well behind, with accuracy between 0.24 and 0.54.

The clearest difference is the non-structural partition walls. The reference model has 90 blockwork partitions, about 365 m in total. Most of them don’t sit on grid lines, so each one has to be found on the architectural plan individually. Opus and GPT-6 built 70 to 80 percent of them; Kimi and DeepSeek built about a third.

Figure 3. Share of partition walls built, by length, averaged over the three toolchains.

The plan below shows DeepSeek’s IFC result, with each reference wall colored by what the model did with it. Red walls are missing: about 285 m, including roughly three quarters of the 200 mm internal partitions. Orange walls were built with the wrong thickness: about 247 m, mostly because every 300 mm external wall was modeled at 400 mm. When I went through the model element by element, the pattern was that it did look at every drawing but applied values read at a few spots to everything: all 100 beams share the same top elevation, and the pile caps use only three sizes. Nothing was checked beam by beam or wall by wall. Its BimDown run shows the same habit: external walls at 350 mm, and pile caps that, in the model’s own summary, were “assigned by rule, not checked one by one”.

Figure 4. DeepSeek V4.1 Flash (IFC): walls compared with the reference model. Grey is correct, orange has the wrong thickness, red is missing.

Kimi failed in a different way. In its IFC run, the helper function it wrote placed each box by its corner instead of its center, so every column, pile cap and pour strip ended up shifted by half its width; a 3 m pile cap moved by 1.5 m. The drawing reading was mostly right, but on site this would mean setting out a whole row of foundations in the wrong place. This counted as an obvious failure and was retried once; the retry did worse.

Opus and GPT-6 made mistakes too, concentrated on dimensions that have to be checked one by one. In its IFC run GPT-6 found all 72 columns, but 36 of them have the wrong section, usually off by one 100 mm step (700×700 built as 600×600, for instance); in its BimDown run the count was 48. Opus’s Revit model has 24 columns with the wrong section, and its wall-thickness errors are mostly 300 mm walls built at 200. In this drawing set the columns are drawn as outlines without dimensions, so their size has to be scaled off the drawing. These are reading-precision errors rather than problems with the modeling code or engineering logic.

Figure 5. GPT-6 Astra (IFC): every column is in the right place; red ones have the wrong section size (enlarged for visibility).

Secondary-structure detailing: with explicit rules, all four do well

When the rules are spelled out (constructional columns no more than 5 m apart, running from the base of the wall up to the underside of the beam or slab above, and so on), all four models did well. Taking each model’s best toolchain, average compliance with the five rules was 96 to 97 percent for every model. Every opening in every run got a valid lintel or ring beam.

The lost points cluster in two places. Constructional columns at free wall ends were often left out; even Opus scored only 0.60 on that item in its IFC and Revit runs. Column height was the other: in DeepSeek’s Revit run only about half the columns had the right height, and 35 of them ran up into the beam or slab above (by a median of about 520 mm). Getting this right means looking up the underside of whatever sits above each column, and the model didn’t. Kimi’s Revit run was also clearly worse than its other two (0.77 against about 0.96), mostly because walls longer than 5 m were not given intermediate columns.

4D scheduling: rules met, judgment missing

All four models produced schedules that satisfy all ten scheduling rules on all three toolchains, with total durations between 99 and 155 days. In 11 of the 12 runs the tower-crane plan kept the 8 m clearance between jib and structure for the whole programme, usually with a minimum clearance of only 8.4 to 10.9 m. The exception was Kimi’s BimDown run, which fell short of 8 m on 15 days.

The difference is in how elements were assigned to tasks. The tower’s concrete core contains 787 concrete beams and 157 concrete columns, 944 elements or about 30 percent of the tower. They are part of the core and are built with the core walls. Opus put them in the core task on all three toolchains. GPT-6 got this right only on BimDown (95 percent) and filed the beams, or beams and columns, under “other” with IFC and Revit. Kimi got it wrong on every toolchain. DeepSeek on BimDown scheduled 740 core concrete beams as steel beams, so concrete beams would follow the steel erection sequence, which is the wrong order. Getting this right means telling concrete beams from steel beams by material and location and knowing how a concrete core is built. The prompt doesn’t say so; it is engineering common sense. One relevant difference between formats: BimDown’s beam table has no material column, so concrete and steel beams can only be told apart by their names, whereas IFC carries material information.

Figure 6. Share of the concrete beams and columns inside the core that were scheduled with the core walls, for each toolchain.

Other findings

Image vs. CAD file. I reran the seven reading groups with the original CAD file (DXF, queryable by layer, text and coordinates) instead of an image. Scores were about the same (Figure 2), slightly higher for Kimi and DeepSeek. The CAD route took far fewer turns: 14 per question group on average, against 29 with images. Opus went from 130 turns to 50 across the seven groups, DeepSeek from 513 to 211. If you have the CAD file, give the model the CAD file.

Toolchain. The toolchain matters somewhat, and in different ways for different tasks. For modeling, BimDown gave the best results for most models: all four scored higher on both detection and accuracy with BimDown than with IFC, and for Opus, GPT-6 and DeepSeek it was the best of the three overall. Revit also beat IFC on detection for all four, with no consistent pattern in accuracy. In detailing, Kimi was clearly worse in Revit. For 4D assignment there was no consistent pattern. Opus got similar results on all three toolchains, while the other models varied more. In practice, with a strong model you can stay with whatever workflow you already have, while with a weaker model it is worth picking the toolchain per task.

Figure 7. Modeling score (mean of detection and accuracy) for each toolchain.

Revit was not reliably the most expensive route either: comparing the same model on the same task, Revit cost more than the other two in 6 of 12 cases. The largest gap was GPT-6 on modeling, about $21 per run in Revit against about $7 with IFC or BimDown. BimDown runs cost about the same as IFC runs.

Reasoning effort. The main results all use “high”. I also have one round at a lower setting for Opus (medium) and GPT-6 (low), and an extra round at “max” for DeepSeek. Most scores moved by less than 0.05. Going from medium to high, Opus improved clearly in one of nine comparisons (IFC modeling, 0.85 to 0.90). GPT-6 improved clearly in one of five (4D assignment on BimDown, 0.87 to 0.99). DeepSeek at “max” did better in three comparisons, all in detailing and 4D, and worse in two, both in modeling. On these tasks the effort setting is not what decides the outcome.

Cost. At API list prices, one run of the basement modeling task cost about $11 with Opus, $12 with GPT-6, $3.40 with Kimi and $0.35 with DeepSeek, a spread of more than 30x. Kimi’s and DeepSeek’s models are still far from usable, though, so the low price doesn’t buy much yet. Most modeling runs took between 15 and 45 minutes.

Figure 8. Cost of one basement modeling run at API list prices, averaged over the three toolchains.

Limitations

• One run per combination. Modeling scores vary noticeably between reruns of the same configuration: DeepSeek on BimDown, run three times, ranged from 0.20 to 0.54 in accuracy. Differences smaller than about 0.2 in modeling, or 0.1 in the other tasks, shouldn’t be read as rankings. The modeling gap between Opus/GPT-6 and Kimi/DeepSeek is at least 0.35, outside that range.
• Few test cases. Modeling, detailing and 4D each have a single main case, all from the same kind of public building.
• Model plus harness. Tool design and context management in each harness affect the outcome, so the results can’t be attributed to the model alone. “High” effort is nominally the same setting across vendors but not strictly equivalent.
• A new format. BimDown is recent, so the models have probably seen little of it in training.
• One generated drawing. Of the five modeling drawings, four are real construction drawings from a design institute (the kind of firm that produces construction documents in China). The beam drawing in flat notation was generated from the reference model.
• Ceiling effects. The reading questions don’t separate the models much, and every run passed all ten 4D scheduling rules, so those two measure baseline ability more than relative strength.
• Costs are list-price estimates, intended only for comparison between models.

What this means

Can be handed to a model today: answering specific questions about a construction drawing; rule-based detailing such as secondary structure; and producing a schedule that meets stated constraints and linking it to the model. All four models handle most of this, and spot checks are enough.

Needs a human reviewer: building a full level from drawings. Opus and GPT-6 produce a first draft with the main elements essentially in place, which saves a lot of modeling time, but wall thicknesses, column sections and door/window tags still need checking item by item. Kimi and DeepSeek miss too many walls and get too many sizes and levels wrong to be useful for drafts yet.

Not yet safe to delegate: judgments that need engineering knowledge the prompt doesn’t spell out, such as which crew builds the concrete members inside the core, how a constructional column meets the beam above it, or how crane climbing keeps pace with the structure. Only Opus got the core assignment right on all three toolchains, GPT-6 on one, and errors of this kind are hard to catch without an answer key.

Overall, frontier models can already do a meaningful share of real BIM work inside real BIM software. Their main weakness is item-by-item verification: they look at every drawing, but they don’t measure every wall and every beam. The next things I want to see are whether models can find and fix these errors on their own, and whether the results hold up across more varied projects.