Skip to content
OpenClaw 不踩坑恶意 Skills ,企业需 Skills Registry:Nacos 3.2 发布Know more

Upgrade Manual

1. Version Upgrade Compatibility

This document corresponds to Nacos 3.3.x. Upgrade compatibility for Nacos 3.3.x is as follows:

Nacos VersionUpgrade SupportedRemarks
0.X ~ 1.XNo0.X ~ 1.X versions must first be upgraded to 2.0 or above. Please refer to the Nacos 2.0 Upgrade Guide to upgrade to 2.0 or 2.1 before proceeding.
2.0.X ~ 2.5.XNo direct smooth upgradeNacos 3.3 removes the Config compatibility migration used by 2.x. If the default namespace was used, first upgrade to 3.0.x–3.2.x and verify that its Config data remains queryable, then upgrade to 3.3.x. See Upgrading from 2.x.
3.0.X ~ 3.3.XYesUpgrading from 3.0.X or later to 3.3.X is supported, but the database schema has changed. Compare the schema file for your target database before upgrading and apply the required schema changes first.

1.1 Client Compatibility

Compatibility between Nacos 3.x server and client versions is as follows:

Client VersionCompatibleRemarks
0.xNo-
1.xNoTo continue using 1.x clients, please integrate nacos-api-legacy-adapter yourself.
2.xYes-
3.xYes-

2. Upgrade Steps

2.1. Pre-upgrade Checks

Complete these preparations before upgrading a production cluster:

  1. Confirm that the source cluster version is supported and that the database schema meets the requirements of the target 3.3.x release. See section 2.2 for database checks and changes.
  2. Back up the database, each node’s data, conf, and cluster.conf, and any custom plugins and configuration.
  3. Rehearse the upgrade and rollback in staging with a copy of production data.

2.1.1. Confirm the Upgrade Path from 2.x

Nacos 3.3 removes the Config compatibility migration and no longer supports a direct smooth upgrade from 2.x. After a direct upgrade from 2.x to 3.3.x, Config data previously stored in the default namespace will no longer be queryable.

Before upgrading, confirm that the deployment has not used Config data in the default namespace. If it has, or if this cannot be confirmed, first upgrade to 3.0.x–3.2.x and verify that the default-namespace Config data remains queryable, then upgrade to 3.3.x.

2.2. Apply Database Changes

Before starting any 3.3.x node, first follow the 3.2.x Upgrade Guide to apply the database prerequisites needed to bring the current source schema up to the 3.2.x schema level; this does not require starting a 3.2.x server. Then compare the resulting actual schema with the complete matching schema in the target 3.3.x distribution and convert the remaining differences into reviewed incremental DDL. Never execute a complete initialization file containing CREATE TABLE or DROP TABLE statements directly against an existing database.

For PostgreSQL deployments upgrading directly from 3.0.x, 3.1.x, 3.2.0, or 3.2.1, or deployments that skipped the 3.2.2 schema normalization, first complete the tenant_id not-null migration documented in the 3.2.x Upgrade Guide.

The target distribution provides mysql-schema.sql, pg-schema.sql, oracle-schema.sql, and derby-schema.sql under conf. After applying the 3.2.x schema prerequisites, pay particular attention to these additional 3.3.x changes:

ChangeScope and action
permissions.resource expands to 512 charactersThe default auth and visibility plugins need a longer column for canonical resource identifiers. If the current column is shorter than 512 and explicit visibility grants will be used, run the matching *-upgrade-visibility-permission-resource.sql from the target distribution. Before running the MySQL script, verify that the InnoDB page size, row format, and unique-index length support VARCHAR(512) utf8mb4.
New AI resource search relational tablesNacos 3.3 adds ai_resource_search_document, ai_resource_search_chunk, and ai_resource_task. AI Resource Search is controlled by nacos.ai.resource.search.enabled and is enabled by default. Before starting the target version against an existing external database, extract and review only the required CREATE TABLE and index statements for these tables from the target main-datasource schema; never copy adjacent DROP statements. If any table already exists, prepare an ALTER or data-migration plan instead of dropping and recreating it. These tables are not required only when AI Resource Search is explicitly disabled; Search is then unavailable, and ARD, which depends on Search, cannot be enabled.
PostgreSQL pgvector storage (optional)Only when enabling the default PostgreSQL vector plugin, initialize pgvector objects and ai_resource_search_embedding_pg in the PostgreSQL datasource that stores embeddings. conf/pg-ai-vector-schema.sql is separate from the main pg-schema.sql, contains CREATE EXTENSION and destructive initialization statements, and requires the pgvector extension and sufficient database privileges. Only a fresh vector store may run it unchanged after review; existing tables require a backup and an incremental migration instead of rerunning the file.
AI resource description column expansionIn PostgreSQL, Oracle, and Derby, expand ai_resource.c_desc and ai_resource_version.c_desc to 2048 when their current length is smaller. Generate reviewed ALTER statements from the actual table definition and target schema instead of assuming the default length from a particular 3.2.x patch.
MySQL collation for fresh schemasFresh 3.3 MySQL schemas use the case-sensitive utf8mb4_bin collation for existing core tables. No general in-place collation migration is currently provided for existing databases. Do not bulk-alter production tables from the full schema; review case-sensitive identity requirements and use a separately validated migration plan if alignment is required.

2.3. Migrate Configuration and Plugins

Nacos 3.3 unifies server plugin configuration. Most legacy configuration keys remain supported in 3.3, but may be removed in a future release. They do not all need to be replaced during this upgrade. Compare the current configuration with the target distribution’s conf/application.properties and migrate progressively to the new keys; avoid overwriting the target file with the source-version configuration file. See System Configurations for commonly used parameters and Plugin Migration Guide for migration guidance.

2.3.1. Default Auth Plugin

The default implementation remains nacos, but it is managed as auth:nacos in the unified plugin system. Its selector and private properties move to the canonical nacos.plugin.auth.* namespace.

3.0.x–3.2.x legacy property3.3 canonical property
nacos.core.auth.system.typenacos.plugin.auth.type
nacos.core.auth.caching.enablednacos.plugin.auth.nacos.caching.enabled
nacos.core.auth.plugin.nacos.token.*nacos.plugin.auth.nacos.token.*
nacos.core.auth.nacos.anonymous.ai.enablednacos.plugin.auth.nacos.anonymous.ai.enabled
nacos.core.auth.ldap.*nacos.plugin.auth.ldap.*, using canonical kebab-case item names from the plugin definitions
nacos.core.auth.plugin.oidc.*nacos.plugin.auth.oidc.*

nacos.core.auth.enabled, nacos.core.auth.admin.enabled, nacos.core.auth.console.enabled, and nacos.core.auth.server.identity.* remain core auth gates or server identity properties. Do not move them into a plugin-private namespace.

When a canonical property and a legacy property are both present, the canonical property wins. An empty canonical value still suppresses the legacy property. Most legacy properties remain readable in 3.3, but the target distribution may already provide defaults for their canonical replacements. When merging configuration, ensure that those defaults do not override the values in use. The startup script automatically handles only a valid legacy token secret in application.properties; other legacy keys are not rewritten automatically and can be migrated progressively by following log warnings. LDAP, OIDC, and custom auth deployments should also review the Auth Plugin page item by item.

If anonymous AI access is enabled, an explicit empty or invalid Authorization, accessToken, username, or password is rejected instead of falling back to anonymous access. Anonymous callers must omit credentials completely; authenticated callers must send valid credentials.

2.3.2. Other Plugin Configuration

Domain3.0.x–3.2.x legacy configuration3.3 canonical model
Datasource dialectspring.sql.init.platform; the older spring.datasource.platform has been removednacos.plugin.datasource-dialect.type
Datasource connectiondb.* and JVM property QUERYTIMEOUTnacos.plugin.datasource.db.*
Traffic controlnacos.plugin.control.manager.typenacos.plugin.control.type
Config Changenacos.core.config.plugin.{pluginName}.*nacos.plugin.config-change.{pluginName}.{itemKey} and unified implementation state
Visibilitynacos.plugin.visibility.typeThis deprecated RESTART selector still chooses the implementation requested by the AI domain and contributes to initial state; nacos.plugin.visibility.{pluginName}.enabled or persisted state determines availability, while nacos.plugin.visibility.enabled remains the capability gate
AI PipelineComma-separated nacos.plugin.ai-pipeline.type and legacy camel-case itemsPer-implementation nacos.plugin.ai-pipeline.{pluginName}.enabled, canonical kebab-case items, and unified state
AI Resource Importnacos.ai.resource.import.enabled, nacos.plugin.ai.importer.*, and Source/preset/list modelsnacos.plugin.ai-resource-import.enabled, nacos.plugin.ai-resource-import.{pluginName}.*, and fixed managed sources

Legacy properties may remain readable as aliases during the compatibility window and produce migration warnings, but a present canonical property has higher priority. Persisted implementation state also overrides static .enabled initial values. Dialect and exclusive implementation selectors, and every field marked RESTART, must be changed through static configuration followed by a restart, not through the plugin configuration API.

AI Resource Import now defaults to enabled when neither its canonical gate nor the legacy alias is configured. To keep it disabled after the upgrade, set this before rollout:

nacos.plugin.ai-resource-import.enabled=false

The legacy Importer/Source SPI, clonable Source/preset model, and configuration-based cloning of one importer to multiple endpoints have been removed. External implementations must migrate to AiResourceImportServiceBuilder. See AI Resource Import migration for the detailed mapping and built-in sources.

2.3.3. Custom and Third-party Plugins

  • Record each implementation by its stable pluginType:pluginName, remove duplicate identities, and confirm that the target release discovers the intended JAR. Duplicate identities now resolve deterministically with the first implementation winning.
  • Older zero-configuration plugins usually remain binary-loadable, but they appear configurable only after implementing PluginConfigSpec definitions, a current configuration snapshot, and apply callbacks.
  • Older Config Change binaries remain loadable during the compatibility window but show configurable=false; migrate them to definitions and callbacks.
  • AI Pipeline no longer loads PublishPipelineServiceBuilder. Recompile pipeline plugins, register PublishPipelineService directly with a public no-argument constructor, and implement the unified configuration contract.
  • Third-party datasource plugins that implement or register the removed ConfigInfoBetaMapper, ConfigInfoTagMapper, or ConfigMigrateMapper SPIs must be rebuilt without those interfaces. Complete pre-3.0 data migration before upgrading.
  • The legacy AI Resource Import SPI has no adapter. Recompile the plugin and update its configuration and callers together; do not mix the two models.
  • Critical plugin types such as auth, datasource dialect, and AI storage can block startup when their selected implementation is missing, disabled, or fails initialization. Validate selection, state, and initialization in staging.
  • On one node, verify plugin list/detail, effective sources, masking, and RESTART guidance. Do not submit runtime configuration understood only by 3.3 nodes during a rolling upgrade.

2.4. Distribution Upgrade

2.4.1. Download the Target Version

Go to the Nacos download page, choose a stable version, and click ${nacos.version}.zip in the Binary Package column.

If the download is rate-limited, retry later or use GitHub instead.

2.4.2. Prepare a New Installation Directory

Extract the target distribution into a new directory:

Terminal window
unzip nacos-server-${target_version}.zip -d ${INSTALL_PARENT}
# or tar -xvf nacos-server-${target_version}.tar.gz -C ${INSTALL_PARENT}

The 3.3 distribution contains a coordinated server JAR, default plugins, schemas, startup scripts, and configuration templates. Do not replace only target/nacos-server.jar, and do not overwrite the new bin, conf, or complete plugins directory with files from the source version.

Migrate deployment-specific data as follows:

  • Start from the new application.properties and move the properties verified in section 2.3 one by one.
  • Compare and recreate cluster.conf, JVM options, log paths, certificates, and other deployment-specific files.
  • Copy only custom or third-party plugin JARs that passed 3.3 compatibility verification.
  • Every deployment must migrate or restore that node’s existing data/plugin so plugin-states.json and plugin-configs.json are retained. With Derby or embedded storage, migrate the complete data only after shutdown and a full backup. An external-database cluster must not blindly copy active Raft data between nodes; reuse that node’s own persistent volume or follow the rehearsed node-recovery procedure. Never let old and new processes use the same data directory concurrently.

2.4.3. Restart or Roll Through the Cluster

For a standalone deployment, stop the old instance and start with the new distribution scripts:

Terminal window
${OLD_NACOS_HOME}/bin/shutdown.sh
${NEW_NACOS_HOME}/bin/startup.sh -m standalone

For a cluster, upgrade one node at a time. Stop one source-version node, start it with its 3.3 configuration and the new distribution, and wait until it rejoins the cluster and core behavior is healthy before proceeding to the next node. Do not change unified plugin runtime state or configuration until all nodes run 3.3.

On Windows, use shutdown.cmd and startup.cmd from the new distribution rather than reusing the old scripts.

2.5. Docker/Kubernetes Upgrade

Container deployments must also complete the database, historical migration, and plugin checks in sections 2.1 through 2.3 before changing images. Compare the target image’s environment-variable conversion rules and the mounted application.properties; do not assume that every old environment variable maps to a canonical plugin property. See System Configurations - Startup Script and Image Variables.

Update the image version in the Compose file:

services:
nacos:
image: nacos/nacos-server:${target_version}

Pull the target image. For a standalone service named nacos, rebuild only that service:

Terminal window
docker compose pull
docker compose up -d --no-deps nacos

When Compose manages a multi-node cluster, do not run one up operation that recreates every Nacos service. Run docker compose up -d --no-deps ${nacos_service_name} for one service at a time, verify that the node has rejoined and is healthy, and then continue with the next service.

2.6. Wait for Automatic MCP Data Migration

When upgrading a deployment with existing MCP Server data from Nacos 3.0.x through 3.2.x, no manual data migration is required. After the target version starts, a background task scans the existing MCP data and adds the information needed by the new management features. The task retries automatically after a failure.

The migration reuses the existing MCP service, Tools, and Resources configuration. It does not move or rewrite this content, and it does not change MCP service addresses, client calls, or gateway access. Existing MCP services can therefore continue serving traffic during the migration.

2.6.1. Expected Behavior During the Upgrade

  • During migration, Nacos marks its internal state as SYNCING. This means the background task is still running or waiting for nodes and search indexing to become ready. It is not an operating mode that users must select; normally, continue the upgrade and wait for automatic retries.
  • Existing MCP management APIs remain available until migration completes. New MCP management APIs may temporarily return 409 / RESOURCE_CONFLICT. This normally means background migration is not finished, not that data has been lost. Continue using the existing APIs and inspect Server logs.
  • The migration task runs after the Server starts and retries about every five minutes by default. If the first scan adds or repairs data, another scan is required to confirm the result, so completion may take one or more cycles.
  • Migration completes only after every Server node has been upgraded, the historical-data scan succeeds, and any enabled AI resource search finishes index initialization. Waiting while old-version nodes remain during a rolling upgrade is expected. Finish upgrading all nodes; no manual data or traffic switch is required.
  • A cluster without historical MCP data also completes the check automatically and normally needs no additional action.

See System Parameters for migration settings. Normally keep nacos.ai.mcp.resource.reconciliation.enabled=true. A shorter interval increases Config and database scan load and should be used cautiously only in staging, testing, or controlled troubleshooting.

2.6.2. Do Not Operate on Internal Migration Data

Nacos stores internal migration state in Namespace _nacos_internal_ and group nacos_internal. These Config entries are used only by Nacos during the upgrade migration. Users do not need to inspect or maintain them and should not modify or otherwise operate on them. If an error occurs, inspect Server logs first. Perform a read-only query only when authorized for further troubleshooting, and do not change the content.

2.6.3. Troubleshoot an Incomplete Migration

SymptomRecommendation
New MCP management APIs continue to return 409 / RESOURCE_CONFLICTContinue using the existing APIs, confirm that all Server nodes are upgraded and healthy, inspect migration-related logs, and wait for the next automatic retry.
Logs report that the search index is not readyIf AI resource search is enabled, confirm that the search tables required by section 2.2 are available and inspect its startup and indexing logs. Do not disable search temporarily just to complete migration.
Logs report a Namespace or historical MCP data scan failureCheck the Namespace, database, and Config services. Common causes include duplicate MCP names or IDs, missing versions, or missing, malformed, or conflicting Server, Tools, or Resources configuration.
Logs report that another node is running the migrationThis is normal cluster coordination. Do not take over manually or delete internal data; wait for the current node to finish or for an automatic retry.

After migration completes, the new MCP management APIs become available and Server logs record completion. Before repairing historical data, back up the database and related Config data and retain complete logs. Use supported management APIs instead of deleting database rows or internal migration data directly. If the correct repair is unclear, preserve the evidence and contact Nacos maintainers.

2.6.4. Rollback and Downgrade Boundary

After MCP historical-data migration completes, the new management state is not automatically reversed when the service is downgraded. Do not allow old-version nodes to handle MCP management requests, and do not attempt rollback by deleting internal migration data or newly added database rows.

Existing MCP service content and calling patterns remain compatible, but subsequent management operations should be handled by target-version nodes. For an emergency downgrade, retain backups and logs, isolate MCP management requests, and work with maintainers on a verified recovery plan.

2.7. Automatic Historical A2A Data Migration

This section applies when upgrading from Nacos 3.1.x through 3.2.x with historical A2A Config definitions, Version content, or old A2A SDKs still publishing Runtime Endpoints to <legacyEncodedAgentName>::<exactVersion> Naming services. The target release migrates those definitions to canonical Agent Resources, Versions, and Agent Storage, and converges live Runtime Endpoints into the canonical RAD Runtime.

Migration does not start by default, and there is no separate migration enable switch. Before the rolling upgrade, every target-version node must explicitly use the same configuration:

nacos.ai.a2a.compatibility.mode=AUTO
nacos.ai.a2a.migration.legacy-naming-shadow-enabled=false

The default CANONICAL mode uses only canonical Agent/RAD and does not scan historical A2A data. During the rolling upgrade, every new node must also use the same available Agent Storage provider and the same shadow policy. Once a migration plan is created, the effective Storage provider and shadow choice are frozen in its marker; inconsistent node policies prevent reconciliation and cutover.

legacy-naming-shadow-enabled=false means that after cutover to CANONICAL, Nacos no longer promises to materialize the old exact-Version Naming service. Ordinary legacy A2A APIs and SDKs query, subscribe, and publish Endpoints through the compatibility layer, so they do not require the shadow. If an application bypasses the A2A/RAD APIs and consumes the <legacyEncodedAgentName>::<exactVersion> serviceName directly through the Naming Gateway, assess that dependency and select true before migration begins. Do not treat a local property change after marker creation as a dynamic switch.

The migration-related properties and defaults are:

PropertyDefaultOperational guidance
nacos.ai.a2a.compatibility.modeCANONICALHistorical migration requires an explicit AUTO.
nacos.ai.a2a.migration.legacy-naming-shadow-enabledfalseWhether to retain the old exact-Version Naming shadow after cutover; frozen when migration starts.
nacos.ai.a2a.migration.reconciliation.interval-seconds300Full reconciliation interval.
nacos.ai.a2a.migration.reconciliation.page-size100Number of historical definitions scanned per page.
nacos.ai.a2a.migration.lease-duration-seconds600Renewable lease duration for the migration owner.
nacos.ai.a2a.migration.quiescing-timeout-seconds120Time before one QUIESCING attempt returns to SYNCING.

Except for the AUTO and shadow decisions, normally retain these defaults. Consider lowering them only in staging or controlled troubleshooting. A shorter lease must still exceed one expected reconciliation unit, and a shorter reconciliation interval increases Config, Storage, and database load.

Keep these three compatibility surfaces distinct:

SurfaceMigration and cutover semantics
Legacy A2A APIs/SDKsRead and write historical definitions during SYNCING; after CANONICAL, the canonical Agent/RAD compatibility facade serves them. Migration does not provide request-level dual reads, merged reads, or long-term definition dual writes.
Canonical Agent/RADHolds the migrated definitions, Search, Discover, Watch, and Runtime state and becomes the terminal authority.
Optional legacy Naming Gateway shadowServes only callers that directly read the old exact-Version serviceName. It is not a definition authority and does not produce duplicate RAD Endpoints or Watch events.

2.7.1. Expected Behavior During the Upgrade

StateUser-visible behavior
SYNCINGHistorical A2A Config definitions remain authoritative. Legacy A2A API/SDK queries, subscriptions, and definition writes continue, as do Endpoint registration, heartbeats, and Redo. New nodes idempotently migrate definitions and Versions in the background and publish each still-live Runtime Endpoint to the old Naming layout as primary plus the canonical RAD layout as a required mirror. Background failures retry and do not turn a legacy definition write into dual authority.
QUIESCINGNacos briefly fences A2A definition mutations to complete a consistent, single-authority cutover. Definition reads, Search, RAD Discover/Watch, Endpoint registration, and existing traffic remain available. An HTTP definition mutation returns HTTP 409; the standard Result has code 50105 (AGENT_MIGRATION_IN_PROGRESS) and the detail in data. Legacy SDK/gRPC responses preserve detail code 50105. Callers should retry later. A timeout, member change, or final validation failure automatically returns the plan to SYNCING.
CANONICALThis is the permanent terminal state. Canonical Agent/RAD becomes the definition and Runtime authority, while legacy A2A APIs/SDKs use its compatibility facade. Do not try to roll back by deleting internal state or reintroducing a legacy-only binary. Direct legacy Naming Gateway visibility is determined only by the frozen shadow policy.

The system completes cutover only after every current Server node has been upgraded and reports consistent migration ability and policy; all historical definitions and Versions are complete, readable, and conflict-free in the target Agent Storage; enabled Search indexes are ready; and all live Runtime mirrors and required retries have converged. Remaining in SYNCING while any old node is present during a rolling upgrade, or while any gate is not satisfied, is expected.

2.7.2. Do Not Operate on Internal Migration Data

Nacos stores migration markers, leases, and progress under Namespace _nacos_internal_ and group nacos_internal. These Config entries are for Nacos cluster coordination only and are not user-maintained data. Do not edit, delete, or forge them, and do not try to roll back by deleting the marker.

Inspect Server migration logs first when troubleshooting. Only when authorized and requested by maintainers should you inspect internal state read-only; never write content into this Namespace or group.

2.7.3. Troubleshoot an Incomplete Migration

Failures are isolated per entry, so valid entries can continue migrating. However, any damaged entry, conflict, or unconverged gate prevents global cutover. Use Server logs to locate the cause:

SymptomRecommendation
The plan remains in SYNCING, and logs report inconsistent member ability or policyConfirm that every current Server node is on the target version and healthy. Verify that every new node uses AUTO, the same effective Agent Storage provider, and the same shadow setting. Correct and roll the inconsistent nodes.
Search is enabled, but logs report that indexes are not readyConfirm that the Search tables, plugins, and datasource required by section 2.2 are ready. Inspect Search startup and indexing-task logs and wait for convergence. Do not disable Search temporarily to force migration completion.
Logs report a missing historical summary or Version, malformed JSON, an invalid Agent identity or Version, or a conflict with an existing canonical AgentBack up the affected data, decide which side is the intended business fact, and repair the historical definition or explicitly resolve the name conflict through supported management APIs. Migration does not guess names, Versions, latest, or content and does not overwrite a different canonical Agent.
Agent Storage saved content but it is temporarily invisible, or read-back validation failsCheck availability, connectivity, and configuration for the effective Storage provider on every node. Completion requires the target content to be readable and validated; wait for notification or a later reconciliation retry.
Runtime mirror or shadow operations keep retrying, or an Endpoint owner has not converged after reconnectInspect the old SDK connection, Endpoint heartbeats, Redo, and Server Runtime retry logs. A required RAD mirror that has not converged during SYNCING blocks cutover. A terminal optional-shadow failure does not interrupt canonical RAD but affects the direct legacy Naming Gateway.
Logs report that another node owns the lease, or the owner exited uncleanlyAnother owner is normal single-writer coordination. Wait for it to finish. After an unclean exit, wait for the lease to expire and a healthy node to take over automatically; the default upper bound is about 600 seconds. Do not delete the lease.

Do not delete internal control entries, edit the database directly, or forge state to “force completion.” If business data must be repaired, retain backups and complete logs and use supported management APIs. Preserve the evidence and contact Nacos maintainers when the correct repair is unclear.

2.7.4. Rollback and Downgrade Boundary

Before the marker permanently reaches CANONICAL, you may configure every node uniformly as LEGACY and roll the cluster back. Historical A2A definitions and old Naming remain authoritative after the rollout. Canonical target data already produced may remain; you can later restore consistent AUTO, Storage-provider, and shadow settings and resume migration idempotently.

After CANONICAL, rollback is supported only to a version that understands both the permanent marker and canonical Agent/RAD data. A legacy-only binary must not rejoin. Do not delete the marker, lease, progress, or target data to restore historical authority. Enabling the Naming shadow preserves only old Gateway Runtime visibility; it does not provide definition rollback.

The initial migration does not automatically delete historical A2A Config or perform destructive source cleanup. Retain old data for backup and audit; any later cleanup requires a separately reviewed, recoverable plan.

2.8. Post-upgrade Verification and Rollback Preparation

At minimum, verify the following after the upgrade:

  1. Node versions, cluster membership, and health are correct, with no schema, plugin-loading, or configuration-alias errors in the logs.
  2. Config publish/query, service register/discover, Console login, Client API, and Admin API authentication match the pre-upgrade settings.
  3. Deployments upgraded from 2.x through an intermediate release can still query Config data in the default namespace.
  4. Plugin management reports the expected plugin IDs, states, configuration sources, and restart-required fields, with no implementation enabled unexpectedly.
  5. When AI Resource Search is enabled, all three relational tables exist and index tasks converge. When the PostgreSQL vector plugin is enabled, also verify the pgvector schema and datasource connectivity.
  6. Follow section 2.6 to confirm that automatic MCP migration completes, the new MCP management APIs are available, and every Server node is upgraded and healthy.
  7. For a deployment with historical A2A data, follow section 2.7 to confirm that Server logs record migration completion. Compare legacy A2A API/SDK and canonical Agent queries for the same names, Version sets, latest pointer, enabled state, and AgentCard content.
  8. Verify that Agent Search and RAD Discover/Watch return the expected definitions and Runtime Endpoints. Keep an old A2A SDK Endpoint running to verify heartbeats, then perform one controlled reconnect and confirm that Redo remains discoverable through canonical RAD.
  9. Only when the migration plan enabled the Naming shadow, verify through the Naming Gateway that the old <legacyEncodedAgentName>::<exactVersion> serviceName has the same exact-Version Endpoints as canonical RAD. When shadow is false, do not use old-serviceName visibility as an acceptance criterion.

Keep the old distribution, configuration, plugin JARs, and database backup until the observation period ends. Before rollback, remove runtime plugin overrides understood only by 3.3. Roll back AI Resource Import plugin JARs, configuration, and callers together. Do not reverse new tables or widened columns during an emergency rollback; use the database rollback procedure validated before rollout.