NeuBird for Palo Alto Networks Panorama
Palo Alto Panorama troubleshooting, with the firewall change cited.
NeuBird is the AI SRE. It reads Panorama config logs, security policy and traffic logs beside your application telemetry, Kubernetes and deploy history. When a pushed rule is why a service fails, NeuBird names the commit, the administrator, the device group and the rule, with root cause in under 5 minutes at 94% accuracy.
Panorama 2:04 PM
Traffic alert · policy-deny surge · DG-Prod-East
pa-5450-dc1-a · 212 denies in 1 min to 10.40.12.31:8443
NeuBird 2:08 PM
Root cause: Panorama pre-rule deny-nonstd-tls, pushed to DG-Prod-Eastat 2:02 PM, denies tcp/8443. payments-api has listened on 8443 since v2.41, so checkout-svc can't reach it.
Found in 4m 12s · 16 sources queried in parallel
NeuBird 2:08 PM
Fix staged: disable deny-nonstd-tls in DG-Prod-East, then commit and push.
The hidden root cause
Why is a firewall change so hard to find?The evidence sits outside the incident.
A security policy change pushed from Panorama can break a service in a minute. The on-call engineer sees timeouts and 5xx errors in the application, and the firewall that dropped the traffic shows up nowhere in the alert.
The symptom shows up in the app
checkout-svc returns 5xx and its calls to payments-api time out. APM shows the errors, not the reason, because the packets never reach the service.
The cause lives in another console
The rule, the commit and the deny logs sit in Panorama with the network team, while the incident runs in the application team’s channel.
One push reaches every firewall
A push to a device group lands on every firewall in it at once, and nothing in the alert ties the job to the service that broke.
Resolve · The change
Which change was pushed, by whom, and where?Panorama’s config log already records it.
Panorama’s config log records which administrator changed what and when, and its task manager records each Commit All job and the firewalls it reached. NeuBird reads both. This sample incident starts when admin alee adds a pre-rule that denies tcp/8443 to device group DG-Prod-East, and job 18342 pushes it to four firewalls at 14:02.
- 1
A rule is added
Admin alee adds pre-rule deny-nonstd-tls to device group DG-Prod-East at 13:51.
- 2
Committed to Panorama
The change is committed to Panorama at 13:58, ready to push.
- 3
Pushed to four firewalls
Commit All job 18342 pushes DG-Prod-East to its managed firewalls at 14:02.
- 4
Cited as evidence
NeuBird reads the config log and the job, and cites both in the root cause.
Rule · deny-nonstd-tls · pre-rule
- Source zone
- any
- Destination
- any
- Application
- any
- Service
- svc-tcp-8443
- Action
- deny
- Position
- above allow-app-to-payments
Task manager · job 18342
Pushing…Completed 14:02:41Commit All · push to devices · DG-Prod-East + TS-Prod-East
- pa-5450-dc1-a
- pa-5450-dc1-b
- pa-3440-edge-1
- pa-3440-edge-2
Resolve · The traffic
What did the firewall do to the traffic?Denied by one rule, from the minute of the push.
Traffic logs record the action, the matching rule and the session end reason for every session. Denies for deny-nonstd-tls start at 14:03 and run at about 400 a minute, all to payments-api on port 8443. The last allowed session matched allow-app-to-payments eleven seconds before the push completed.
- 1
The onset matches the push
Denies for one rule start at 14:03, a minute after job 18342 completes.
- 2
One rule, one destination
Every deny names deny-nonstd-tls and payments-api at 10.40.12.31 on port 8443.
- 3
The session end reason
Sessions now end in policy-deny. The last allowed session matched allow-app-to-payments and ended with tcp-fin.
Sessions denied per minute · rule deny-nonstd-tls
2,612 denied since 14:03
Resolve · The root cause
How does NeuBird connect the firewall to the outage?Root cause in under 5 minutes, at 94% accuracy.
NeuBird queries 15+ sources in parallel, Panorama beside Datadog, Kubernetes, GitHub and PagerDuty, and lines the evidence up on one timeline. It names the rule, the device group, the administrator and the job, then stages the rollback for one-click approval. The approved rollback becomes a ServiceNow change, which your network team commits and pushes in Panorama. Every verified fix is remembered, with no bulk telemetry stored, so the next incident that matches this pattern starts from the answer.
- 1
16 sources in parallel
Panorama logs sit beside APM, Kubernetes, deploy history and the page.
- 2
One timeline
The rule, the push, the first deny and the first 5xx line up to the minute.
- 3
Root cause, cited
The rule, the device group, the admin and the job, each linked to its evidence.
- 4
Rollback staged for approval
The approved rollback becomes a change ticket your network team commits and pushes in Panorama.
Sources queried in parallel
16
Panorama config logquerying…pre-rule set · 13:51
Panorama traffic logquerying…2,612 policy-deny
Panorama system logquerying…job 18342 · 14:02
- Datadog APMquerying…payments-api timeouts
- Kubernetes · prod-eastquerying…no restarts, no OOMKills
- GitHubquerying…payments-api v2.41 · port 8443
- PagerDutyquerying…paged 14:06
- Postgres metricsquerying…payments-db healthy
+ 8 more sources
Timeline
- 13:51alee adds pre-rule deny-nonstd-tls (svc-tcp-8443, deny) to DG-Prod-East
- 14:02Commit All job 18342 pushes DG-Prod-East to 4 firewalls
- 14:03pa-5450-dc1-a denies checkout-svc → payments-api on 8443
- 14:03checkout-svc 5xx rate climbs to 18%
- 14:04Panorama alerts on the deny surge; NeuBird starts
- 14:06PagerDuty pages the on-call engineer
Root cause
Confidence: highPre-rule deny-nonstd-tls, pushed to DG-Prod-East at 14:02 by alee (job 18342), denies tcp/8443 from any zone. It sits above allow-app-to-payments, and payments-api has listened on 8443 since v2.41 on Nov 18.
Recommended rollback
Mode: RecommendDisable deny-nonstd-tls in DG-Prod-East, then commit and push to DG-Prod-East. Re-add it later scoped to the untrust zone.
Saved to memory: the conclusion and the fix, no bulk telemetry stored.
Which firewall changes does NeuBird trace?
Security pre-rules and post-rules
A new deny placed above an allow, or a post-rule that now catches traffic no earlier rule allows.
NAT rules
A changed translation that sends a service’s traffic to the wrong address or port.
App-ID content updates
A content release that identifies traffic as a different application from the one the rule allows.
Security profiles
A vulnerability or file blocking profile that now resets sessions a service depends on.
Templates and template stacks
Interface, zone and routing changes pushed to firewalls with a template stack.
Prevent and Operate
What does NeuBird catch before the page?And what does it tidy between incidents?
NeuBird reads the same Panorama data when nothing is broken. It catches silent drift and capacity bottlenecks 30 to 60 minutes before the page, such as a session table filling up, an HA peer out of sync or a pending change that would shadow an allow rule. It also runs the hygiene between incidents: unused rules, drift between device groups that should match, and threat log alerts that only add noise.
Prevent
Before the page
A pending change would shadow an allow rule
Pre-rule deny-legacy-smb is committed to Panorama but not yet pushed to DG-Prod-West. It sits above allow-backup-smb, so the push would deny backups to vault-nas-02.
Session table nearing capacity on pa-5450-dc1-a
Active sessions at 78% of the maximum and rising about half a point a minute. Projected to reach the limit in 41 minutes.
78% of max sessions
HA peer out of sync
The system log on pa-5450-dc1-b reports its running configuration not synchronized with the active peer since 06:40.
Operate
Between incidents
Unused rules in DG-Prod-East
37 security rules have had zero hits in 90 days. NeuBird lists them with their last hit and owner tag for review.
Drift between device groups
DG-Prod-East and DG-Prod-West are meant to match, and 4 rules now differ in service or position.
Noisy threat log alerts
One informational signature raised 2,140 alerts this week in the on-call channel. A severity filter on the log forwarding profile would route them elsewhere.
The connection
How does NeuBird connect to Panorama?One API key, and nothing that commits.
NeuBird queries Panorama in place, with no bulk telemetry stored, and deploys in under one hour. How far it goes on its own is set per action across Suggest, Recommend and Act, with configurable human approval gates.
PAN-OS XML API, one API key
NeuBird calls Panorama’s XML API over HTTPS with a key for an administrator whose role grants configuration, operational, log and report access, and nothing that commits.
Configuration, status and logs
Device groups, templates, security and NAT rules, managed-device and HA status, and the traffic, threat, system and config logs your firewalls forward to Panorama.
Every firewall, through Panorama
NeuBird reaches each managed firewall through Panorama by its serial number, so it needs no direct access to the firewalls themselves.
Changes stay in your process
The connection never commits or pushes. A staged rollback goes to your change process, and your network team commits it in Panorama.
Panorama is one of 50+ integrations NeuBird queries in place. For the rest of the network path, see NeuBird for Zscaler and Cisco SD-WAN, or browse all solutions.
FAQ
Common questions
What does NeuBird do with Palo Alto Networks Panorama?
NeuBird is the AI SRE. Connected to Panorama, it reads firewall configuration, managed-device status and the logs your firewalls forward to Panorama, and correlates them with application telemetry, Kubernetes and deploy history. When a pushed policy change is why a service is failing, NeuBird names the change, the administrator, the device group and the rule, with root cause in under 5 minutes at 94% accuracy.
How does NeuBird connect to Panorama?
NeuBird calls the PAN-OS XML API on Panorama over HTTPS with an API key. You create a Panorama administrator whose role grants XML API access to configuration, operational requests, logs and reports, and nothing that commits, then generate a key for it. Setup takes under one hour.
Can NeuBird change our firewall configuration?
The Panorama connection never commits, pushes or edits configuration. When NeuBird finds a rule to roll back, it stages the change with its evidence for one-click approval and hands it to your change process, where your network team commits and pushes it in Panorama. How far NeuBird goes on its own is set per action, across Suggest, Recommend and Act.
Does NeuBird need access to every firewall?
NeuBird reaches each managed firewall through Panorama by its serial number, so it needs no direct access to the firewalls. Firewalls that Panorama does not manage are not visible to the connection.
Which Panorama logs does NeuBird use?
NeuBird uses the logs your firewalls forward to Panorama and its Log Collectors: traffic logs (action, rule, session end reason), threat logs, system logs such as HA state changes, and config logs that record which administrator changed what and when. It queries them in place, with no bulk telemetry stored.
Does NeuBird store our firewall logs?
NeuBird queries Panorama where the data lives, with no bulk telemetry stored. Its memory holds conclusions, such as a verified root cause and the fix that resolved it, never raw traffic, threat or config logs.
NeuBird for Palo Alto Networks Panorama
The AI SRE that answers the page, then keeps production running.
See NeuBird trace an incident to the Panorama change behind it, with root cause in under 5 minutes and the rollback staged for your team.
