AI-assisted development can shorten the path from a service idea to a working government web application. It does not shorten the path an attacker can take through a public form, an API, an identity provider or a privileged staff workflow. Before asking a coding assistant to implement a feature, map the trust boundaries the feature will cross.
The Australian Signals Directorate's September 2026 Information Security Manual (ISM) software-development guidelines explicitly cover human, AI-assisted, AI-powered and AI-driven development, including web applications. They call for threat modelling in the software development lifecycle (ISM-1238) and for the model to be reviewed as the built system and threat environment change (ISM-2039). Those controls are not a claim that threat modelling first appeared in September; this is a practical way to apply the current guidance to a real service.
Begin with the service, not the diagramming tool
Choose one change: for example, a public application form that lets a person upload evidence and lets authorised staff review a case. In one page, show the browser, application, identity service, upload store, staff console and any downstream case-management API. Mark which actor controls each input and which component decides whether that actor may read or change a record. Include the administrator path and the paths that run without a signed-in user.
Then use the four questions in OWASP's threat-modelling guidance: What are we working on? What can go wrong? What will we do about it? Did we do a good job? A useful model records assumptions and mitigations that a team can later verify; a beautiful data-flow diagram without a testable decision is not the deliverable.
Trace one request across each boundary
- Public input to application. Can someone submit another applicant's identifier, an unexpected file type or repeated requests at scale? Define server-side validation, upload processing and rate controls; test failure cases, not only the happy path.
- Application to identity and authorisation. Is authentication being mistaken for permission to access a particular case? Test object-level authorisation with two real test roles and two separate case records. A successful login must not silently grant cross-case access.
- Application to stored data and downstream APIs. Which fields leave the service, under whose authority, and how is an API failure handled? Give service credentials the minimum necessary scope. Keep sensitive case content out of unnecessary logs and test the unauthorised and unavailable-downstream paths.
- Staff actions to public outcomes. Can a staff member's role change, a stale session or a copied URL expose a decision to the wrong person? Specify audit events, session behaviour and a way to revoke access. Test what happens after a role is removed, not merely after initial sign-in.
These are example threats and tests, not a declaration that every system has these defects. Actual controls depend on the agency's data classification, architecture and risk assessment. For web applications, the same ISM guideline separately references the OWASP Application Security Verification Standard (ISM-0971), secure session cookies (ISM-2063), server-side session management (ISM-2066), HTTPS (ISM-1552) and API security risks (ISM-1851). Use the controls that apply to the agreed system boundary rather than treating this short list as a compliance certificate.
Give the model a change trigger and an owner
A model signed off at project kickoff goes stale when the team adds an upload processor, changes the identity provider or connects an AI summariser to case documents. Attach the threat model to the change record and review the affected boundaries when a feature, integration, data flow or access role changes. For each material threat, record the owner, mitigation, verification test and residual decision. A reviewer should be able to point from the model to a failing-or-passing test and the implemented control.
AI can help propose abuse cases and draft tests, but it does not know the agency's actual permissions or data flows unless the team supplies and validates them. Do not put live personal information or credentials into an unapproved AI tool. The ISM says AI models may augment security testing, not replace traditional testing; their findings should be validated by appropriately skilled people. A human owner must still decide whether the system's trust boundaries and residual risks are acceptable.
A practical first slice for a delivery team
Take one upcoming change and schedule a short walkthrough with a service owner, developer and security reviewer. Produce a one-page boundary map and a small set of abuse cases; turn the highest-risk cases into automated authorisation or integration tests where feasible. Re-run them before release and after material boundary changes. This is more useful than a fleet-wide threat-model spreadsheet no one opens again.
NAITEC's government digital services include custom web applications and platform integrations. If your team is adding a public form, staff workflow or AI-assisted feature, we can help identify the trust boundaries, build safeguards and test how they hold up in real user and staff journeys. Start with one upcoming change and turn its highest-risk paths into tests your team can run before release.