Release Notes

Note: SDK v0.3.0 is compatible with Amorphic v3.1.2

Features (12)

  1. Dataset Management & Discovery – Extensive capabilities have been introduced for end-to-end dataset lifecycle management. Users can now list, create, search, and retrieve comprehensive details for datasets. Administrators and owners can complete dataset registrations, perform bulk metadata updates, batch delete datasets, and securely export dataset contents directly to S3. Additionally, users can seamlessly query the dependent resources associated with any given dataset.

  2. Advanced File Operations & Analytics – Dataset file handling has been vastly expanded. Users can generate S3 presigned URLs for standard and sample file uploads (to extract schema) and upload files directly from query results. The system supports listing, downloading, and searching the metadata of dataset files, as well as deleting files from the LZ bucket. Furthermore, users can access file-level analytics, list available models for specific files, run models, check their run statuses, and download execution logs.

  3. Domain & Data Classification Governance – New governance tools provide better organization and categorization of data assets across the platform. Users can create, list, retrieve details, update descriptions, and delete distinct business domains. Similarly, data classifications can be created, managed, and deleted to ensure structured grouping and improved discoverability of sensitive or categorized information.

  4. Data Permissions, Roles & Resource Sharing – Granular security enhancements allow for finer control over platform access. System administrators can create, list, update, and delete access roles. Resource owners can securely grant (owner/editor/read-only) or revoke access for users and tags, and verify file access lists. Furthermore, robust data permission endpoints allow owners to enforce, update, and delete strict row-level and column-level access controls, including updating specific column tags.

  5. Data Engineering, Iceberg & Reload Operations – Advanced data operations have been introduced to support modern table formats and data synchronization. Users can trigger specific Iceberg/Hudi data load jobs and manage or delete pending Iceberg files. The update also allows users to sync partitions for LF and S3-Athena datasets, refresh materialized views, trigger standard file reloads, retrieve data reload histories, and clear pending reload files.

  6. Platform Telemetry, Auditing & Lineage – Visibility into platform operations is significantly improved with expanded telemetry and lineage tracking. Users can create, update, and retrieve dataset lineage to track historical data relationships, as well as delete dataset lifecycle policies. The platform now exposes system information, global resource searches, schedule job listings, and endpoints to fetch specific resource IDs. Administrators can retrieve comprehensive system audit logs, monitor data load statistics across targets, and view or update file data load throttling settings.

  7. AI Models & Configuration – Integration with foundational AI models has been implemented to support generative capabilities. Administrators can list available AI models, retrieve specific model details, update default parameters, and easily enable or disable models. The system also supports updating configurations specifically for Bedrock models tied to various platform components.

  8. AI Playground (Chats & Notes) – A new interactive environment allows users to experiment with AI models securely. Users can create, list, retrieve, update (title/keywords/summaries), and delete dedicated AI chat sessions. To provide context to models, users can upload URLs to attach files to chats, list associated files, and delete them. The playground also includes endpoints to retrieve paginated chat messages, manage global chat configurations, and maintain personal AI notes (create, read, update, delete).

  9. AI Knowledge Bases – Centralized components have been added to build and manage AI knowledge bases using Bedrock and DynamoDB. Users can create, list, retrieve, update (names/descriptions/guard rails), and delete comprehensive knowledge bases. Workflows are supported to add datasets or domains as resources, update resource descriptions, remove resources, and sync files present inside the datasources. Users can also retrieve knowledge base run metrics and directly query the knowledge base.

  10. AI Agents – The platform now supports the creation and lifecycle management of autonomous AI agents designed to handle complex generative tasks and orchestrations. Users with appropriate access can create new agents, list available agents, retrieve specific agent details, update agent configurations, and securely delete agents when no longer needed.

  11. AI Guard Rails & Security – Robust safety and compliance features have been introduced to ensure AI interactions remain secure. Administrators can create, list, retrieve, update, and delete AI guard rails. Global guard rails can be set as the system default, and users can test chat messages directly against specific guard rails for content filtering. Additionally, administrators can configure, list, and delete component-specific guard rails to strictly control which filters apply to particular agents or knowledge bases.

  12. Developer Tools & Payload Generation – To streamline the programmatic creation of Amorphic resources, new developer utilities have been introduced. Users can generate specific payloads required for creating different resources (e.g., dataset creation payloads) and request presigned URLs to upload files necessary for payload generation workflows.