Plugin System

Internals of the plugin loader, worker runtime, and database entities

Plugin System Internals#

How Hay Core discovers, loads, and runs plugins

This document is for developers working on the plugin system infrastructure itself — the loader, the worker runtime, the database entities, and the hook dispatch. If you're building a plugin, see Getting Started with Plugins instead.

Table of Contents#

• Architecture Overview
• Plugin Discovery and Registration
• The Worker Runtime
• Instance Lifecycle Management
• Database Entities
• Lifecycle Hooks
• Cron Scheduling
• Webhook Routing and Channel Delivery
• The Plugin API (Worker → Core)
• Dashboard-Facing tRPC Surface
• Known Gaps and Legacy Paths

Architecture Overview#

The core model in one paragraph: a plugin is a directory under plugins/core/<name>/ (or plugins/custom/{organizationId}/<name>/) whose package.json contains a hay-plugin block. The plugin ID is the npm package name (e.g. hay-plugin-klaviyo, hay-channel-instagram-meta). Plugins do not run inside the core process — for each (organizationId, pluginId) pair, core lazily spawns a separate Node.js HTTP worker via the SDK runner, talks to it over localhost HTTP, and kills it after 5 minutes of inactivity.

There is no manifest.json. The plugin system was migrated to a TypeScript-first model where all metadata lives in package.json → hay-plugin and in the code itself (via defineHayPlugin from @hay/plugin-sdk). Nothing in core reads a manifest.json file. Older docs and comments that mention one are stale.

Key services#

The SDK itself lives at packages/plugin-sdk/ — types under packages/plugin-sdk/types/, the worker runner under packages/plugin-sdk/runner/.

Plugin Discovery and Registration#

File: server/services/plugin-manager.service.ts (PluginManagerService)

Initialization flow (initialize())#

1. discoverPlugins() — scan plugins/core/ and every plugins/custom/{organizationId}/ directory.
2. For each subdirectory, registerPlugin(pluginPath, sourceType, organizationId):
   • Read package.json. If there is no hay-plugin key, the directory is silently skipped — this is the sole discovery criterion.
   • Plugin ID = package.json.name. Display name = hay-plugin.displayName or a Title-Cased version of the package name with the hay-plugin- prefix stripped (parseDisplayName).
   • Build a manifest from the hay-plugin block and persist it as the manifest jsonb column. The primary type comes from hay-plugin.category, plus extra types inferred from capabilities (inferTypeFromCapabilities: mcp → mcp-connector, routes/messages → channel, sources → retriever, products → products; if nothing matches, the inferred list defaults to ["mcp-connector"]) and documentImporter: true → document_importer. The hay-plugin.env allow-list is stored on the manifest as permissions.env, and capabilities is mirrored into permissions.api.
   • Load i18n JSON files from the plugin's i18n/ directory into manifest.i18n.
   • Calculate a SHA-256 checksum over the plugin's .ts/.js files (excluding node_modules and dist) for change detection (calculatePluginChecksum).
   • Upsert into plugin_registry via pluginRegistryRepository.upsertPlugin() and the in-memory registry map.
3. loadRegistryFromDatabase() — merge DB rows into the in-memory map.
4. validateExistingPlugins() — plugins present in the DB but not found on disk are marked status = "not_found".
5. restorePluginsFromZip() — custom/git plugins with a stored ZIP upload get re-extracted if their directory is missing.
6. initializeAutoActivatedPlugins() — legacy path for autoActivate: true plugins (document importers) that register a tRPC router at boot.

The hay-plugin block#

The fields core actually reads (interface HayPluginBlock in plugin-manager.service.ts):

interface HayPluginBlock {
  displayName?: string;
  category?: PluginType;        // mcp-connector | channel | retriever | playbook | document_importer | system | products
  entry?: string;               // e.g. "./dist/index.js"
  capabilities?: string[];      // routes | mcp | auth | config | ui | messages | customers | sources | products
  config?: ...;                 // config schema (usually declared in code instead)
  env?: string[];               // allow-list of host env vars the worker may inherit → manifest.permissions.env
  auth?: ...;
  channel?: string;             // channel plugins: the channel slug, e.g. "instagram"
  autoActivate?: boolean;       // legacy (document importers)
  trpcRouter?: string;          // legacy (document importers)
  documentImporter?: boolean;   // legacy (document importers)
}

Capabilities are declarative: they drive marketplace classification, type inference, and the scope of the worker's JWT. They are not hard-enforced against what the plugin actually implements.

The manifest stored in the registry is RegistryManifest — identical to HayPluginManifest (server/types/plugin.types.ts) except capabilities is the flat string array from package.json. Consumers branch on Array.isArray(manifest.capabilities).

The Worker Runtime#

File: server/services/plugin-runner.service.ts (PluginRunnerService, singleton via getPluginRunnerService())

There is no in-process plugin loading and no generic "process manager" service — the runner service is the single source of truth for worker processes, keyed by "orgId:pluginId" in an in-memory Map<string, WorkerInfo>.

startWorker(orgId, pluginId)#

1. Load the PluginRegistry row and the org's PluginInstance row; refuse to start if the instance doesn't exist or isn't enabled.

2. If authMethod === "oauth" and the stored token expires within 5 minutes, refresh it via oauthService.refreshToken() before spawning.

3. Allocate a port (server/services/port-allocator.service.ts) and resolve the org's config with resolveConfigForWorker (server/lib/config-resolver.ts), which merges DB config, env: fallbacks, and auth credentials. The config schema used for env fallbacks comes from the cached worker metadata (plugin_registry.metadata.configSchema).

4. Spawn the SDK runner as a child process:

   node packages/plugin-sdk/dist/runner/index.js \
     --plugin-path=<abs path> --org-id=<orgId> --port=<port> --mode=production
   

   (In development, if the compiled runner doesn't exist, it falls back to npx tsx packages/plugin-sdk/runner/index.ts.)

5. Environment injection (buildSDKEnv):

   • HAY_ORG_ID, HAY_PLUGIN_ID, HAY_WORKER_PORT
   • HAY_ORG_CONFIG — JSON: { org: { id }, config: <resolved config> }
   • HAY_ORG_AUTH — JSON-serialized AuthState (decrypted credentials)
   • HAY_API_URL + HAY_API_TOKEN — a JWT (generatePluginJWT) scoped to { organizationId, pluginId, scope: "plugin-api", capabilities }, expiring in 24h, so the worker can call back into core
   • Host env vars from the plugin's allow-list (manifest.permissions.env, i.e. hay-plugin.env) — filtered through a deny-pattern list (SECRET, PASSWORD, TOKEN, PRIVATE_KEY, CREDENTIAL, ^DB_, ^DATABASE, ^REDIS, ^AWS_, ^OPENAI, ^ANTHROPIC, ^JWT, ^SMTP, ^GITHUB, ^NPM, ^PLUGIN_ENCRYPTION, ^HAY_) so secret-looking names are blocked by default

6. Wait for the worker's GET /metadata endpoint to respond (up to 20 attempts × 500ms) — not /health.

7. Update the PluginInstance row: runtimeState = "ready", running = true, processId, health "healthy". On failure: runtimeState = "error" + lastError.

8. Kick off MCP tool discovery in the background (fetchAndStoreTools in server/services/plugin-tools.service.ts, which caches the worker's GET /mcp/list-tools result).

stopWorker(orgId, pluginId)#

POST /disable on the worker (5s timeout, best-effort), then SIGTERM, then SIGKILL after 5 seconds if it hasn't exited. The port is released, the worker removed from the map, and the instance row updated (runtimeState = "stopped", running = false); the process exit handler also updates state/health when a worker dies on its own.

Worker HTTP surface#

The SDK runner's HTTP server (packages/plugin-sdk/runner/http-server.ts) exposes exactly:

All dashboard/webhook traffic to a worker goes through the core proxy: ALL /v1/plugins/:pluginId/* (server/routes/v1/plugins/proxy.ts). The proxy resolves the organization from auth, subdomain, or query param, starts the worker on demand, bumps its activity timestamp, strips credential-bearing headers (authorization, cookie, proxy-authorization, etc.), and forwards the request.

Instance Lifecycle Management#

File: server/services/plugin-instance-manager.service.ts (PluginInstanceManagerService)

• ensureInstanceRunning(organizationId, pluginId) — the on-demand entry point. Deduplicates concurrent startups via a startupQueue map, checks pool limits, and delegates the actual spawn to the runner service.
• Idle cleanup: INACTIVITY_TIMEOUT_MS = 5 minutes. cleanupInactiveInstances() is not run by an internal timer — it's invoked by the platform scheduler job plugin-instance-cleanup (see server/services/scheduled-jobs.registry.ts), which runs every 60 seconds. Activity is tracked both in memory (instanceActivity map) and in the last_activity_at column.
• Pool limits: each plugin's plugin_registry.max_concurrent_instances (default 10) caps concurrent workers across orgs. ensureInstanceRunning waits up to 30 seconds for a slot (waitForAvailableSlot) before throwing.
• updateActivityTimestamp() is called from the proxy on every forwarded request so active workers aren't reaped mid-conversation.
• stopAllForOrganization(organizationId) tears down every worker for an org.

Implication for plugin behavior: workers are ephemeral. Anything a plugin needs long-lived (schedules, webhook subscriptions) must be delegated to core (register.cron, register.webhookRouting) rather than held in worker memory.

Database Entities#

All three entities live in server/entities/ and follow the project's snake_case-in-DB / camelCase-in-TS convention (see server/database/DATABASE_CONVENTIONS.md). The field lists below reflect the entity classes — do not trust older docs showing raw SQL with SERIAL primary keys; IDs are UUIDs.

plugin_registry (plugin-registry.entity.ts)#

One row per discovered plugin (global, not per-org). Unique index on pluginId.

plugin_instances (plugin-instance.entity.ts)#

One row per (organizationId, pluginId) — the org's enablement, config, auth, and runtime state. Unique index on (organizationId, pluginId). Note pluginId here is a uuid FK to plugin_registry.id, not the string plugin ID.

plugin_webhook_routes (plugin-webhook-route.entity.ts)#

Plugin-agnostic index mapping (pluginId, routingKey) → (organizationId, pluginInstanceId), with a unique index on the pair. Populated from the routing keys a plugin returns from its onConnected hook; consumed by the webhook router to resolve which org a shared-webhook payload belongs to. Cascade-deleted with the plugin instance.

Lifecycle Hooks#

A plugin's entry module default-exports defineHayPlugin(factory); the factory returns a HayPluginDefinition (packages/plugin-sdk/types/plugin.ts — the only required field is name). Core drives the hooks over the worker HTTP surface:

The register.* API available in onInitialize (packages/plugin-sdk/types/register.ts): register.config(schema), register.auth.apiKey(...) / register.auth.oauth2(...), register.route(method, path, handler), register.ui.page(...), register.cron(options), register.webhookRouting(descriptor). Everything registered surfaces as data in GET /metadata, which core caches on plugin_registry.metadata.

OAuth is entirely core-side. server/services/oauth.service.ts runs authorization, token exchange, and refresh — including non-standard flows a plugin declares via tokenExchange/tokenRefresh descriptors (packages/plugin-sdk/types/auth.ts), executed declaratively by core (used by the Instagram plugin's ig_exchange_token/ig_refresh_token). Plugins never call OAuth endpoints themselves; the runner also refreshes expiring tokens before worker start (see The Worker Runtime).

Cron Scheduling#

File: server/services/plugin-cron.service.ts

Because workers are idle-killed, plugins must not self-schedule with setInterval/node-cron. Instead, register.cron declares the schedule as data, and core:

1. Reads cron descriptors from the plugin's cached /metadata.
2. Registers one platform-scheduler job per enabled org per declared cron, named plugin-cron:<pluginId>:<orgId>:<cronName> (via schedulerService.registerJob).
3. When a job fires: ensure the worker is running (waking it if needed), then POST /cron/:name so the handler executes in the worker.
4. If the handler called ctx.auth.update(...), the updated credentials come back in the response and core persists them (encrypted) and restarts the worker.

Jobs are unregistered when the plugin is disabled for the org.

Webhook Routing and Channel Delivery#

Inbound: shared-webhook fan-out#

Some providers (e.g. Meta) deliver all events for all customers to one webhook URL that carries no org identifier. A plugin can declare a WebhookRoutingDescriptor via register.webhookRouting (packages/plugin-sdk/types/webhook-routing.ts) — pure data, no code: an HMAC-SHA256 signature spec (header, format: "sha256-hmac", secret source), an optional GET verification-challenge spec, and dot-path extraction rules (routeKeyPath: { itemsPath, keyPath }).

The dispatch point is in the proxy (server/routes/v1/plugins/proxy.ts): a request to .../webhook with no org identifier, for a plugin that declared a routing descriptor, is diverted to webhookRouterService.handle() (server/services/webhook-router.service.ts), which:

1. Answers the provider's GET verification handshake if declared (echoes the challenge param after checking the verify token).
2. For POSTs, verifies the HMAC over the raw body bytes (verifyHmacSha256 from server/services/plugin-route.service.ts).
3. Extracts a routing key per payload entry (routeKeyPath), resolves key → org via plugin_webhook_routes, and fans each entry out to the right org's worker POST /webhook (with credential-bearing headers stripped).

Core never learns which provider it's routing for — all provider knowledge is in the plugin's declaration.

Inbound: normal per-org webhooks and message ingestion#

Webhooks that do identify the org go straight through the proxy to the worker's registered /webhook route. The plugin verifies/filters the event and then calls back into core with messages.receive (see next section). Inbound dedupe is core-side: if the plugin passes metadata.mid, core claims it atomically via Redis SET ... NX EX and drops duplicates (server/routes/v1/plugin-api/trpc.ts); channels that don't pass a mid get no dedupe.

Outbound: channel delivery#

File: server/services/channel-delivery.service.ts

Subscribes to the Redis websocket:events channel:

• message_received events for bot/human-agent messages (web channel skipped) → resolve the plugin by channel slug via pluginManagerService.findPluginIdByChannel(channel) (matching hay-plugin.channel), ensure the org's worker is up, and POST /deliver with { to, content, messageId, conversationId, ... }. The plugin returns { success, providerMessageId?, error? }; on success core stores the provider message ID. Convention: non-retryable provider errors return HTTP 200 with success: false to avoid retry storms.
• conversation_status_changed → pending-human → POST /escalate on the worker (web channel skipped). This is best-effort: a 404 from plugins that don't implement /escalate is silently tolerated.

See Channel Architecture for the plugin-author view.

The Plugin API (Worker → Core)#

File: server/routes/v1/plugin-api/trpc.ts

Workers call back into core over tRPC-HTTP using the HAY_API_URL/HAY_API_TOKEN env vars. Every procedure is a pluginProcedure gated by the worker's JWT (org + plugin + capabilities; server/trpc/middleware/plugin-auth.ts). Current surface:

• messages.receive — input { from, content, channel, metadata?, senderType?, externalConversationId? }; finds/creates the Customer (keyed on external_id per channel) and conversation, adds the message
• messages.send, messages.getByConversation
• conversations.updateStatusByExternalId
• customers.get, customers.findByExternalId, customers.upsert
• sources.register
• mcp.registerLocal, mcp.registerRemote
• products.upsertMany, products.delete (source is the authenticated plugin ID — a plugin can only touch its own products)

The router self-describes as a "simplified initial implementation" — expect this surface to grow.

Dashboard-Facing tRPC Surface#

Files: server/routes/v1/plugins/index.ts (router), server/routes/v1/plugins/plugins.handler.ts (implementations). All procedures are authenticatedProcedure.

• Queries: plugins.getAll, plugins.get, plugins.getInstances, plugins.getUITemplate, plugins.getMCPTools, plugins.getMenuItems, plugins.testConnection, plugins.getPluginTranslations, plugins.oauth.isAvailable, plugins.oauth.status
• Mutations: plugins.enable, plugins.disable, plugins.restart, plugins.configure, plugins.refreshMCPTools, plugins.validateAuth, plugins.oauth.initiate, plugins.oauth.revoke

There is no generic plugins.invokeTool mutation — MCP tool invocation happens inside the orchestrator via the worker's POST /mcp/call-tool.

Non-tRPC HTTP: the catch-all worker proxy ALL /v1/plugins/:pluginId/* (proxy.ts), and plugin UI assets served at /plugins/ui/:pluginName/:assetPath.

Known Gaps and Legacy Paths#

Things a contributor should know exist (or don't) before building on them:

• onEnable is dead code — typed in the SDK but invoked by neither the runner nor core.
• Document importers bypass the worker model. Plugins with autoActivate: true + trpcRouter (e.g. plugins/core/atlassian) load a tRPC router in-process at boot via server/services/plugin-router-registry.service.ts, coupled to @server/* internals. This is the legacy special case, not the pattern to extend.
• onConfigUpdate is also uncalled — the runner exposes POST /config-update, but core applies config changes by restarting the worker (see the hooks table).
• PluginApiClient is duplicated per channel plugin (plugins/core/instagram/src/plugin-api.ts, and likewise in chatwoot, email, whatsapp) instead of being an SDK export.
• Webhook signature format is limited to sha256-hmac in WebhookRoutingDescriptor.
• The instance pool's queuedRequests stat is tracked but never incremented — pool limiting works via polling (waitForAvailableSlot), not a real queue.
• The plugin_instances.status column is legacy; runtimeState is the field the runner actually maintains.

When contributing to any of the above: follow existing patterns in the neighboring services, keep the core plugin-agnostic (never hardcode plugin IDs — resolve behavior from the manifest/metadata), and remove legacy paths outright rather than layering compatibility shims (this codebase is in alpha; breaking changes are preferred over dead code).