Sold by: PathwayÂ
Deployed on AWS
Debian Linux with pre-installed Pathway, an ultra-fast ETL framework. Ideal for data engineers, analysts, and developers seeking efficient data transformation and integration.
Overview
Developed in Rust with a focus on Python developers, Pathway streamlines the entire ML/AI project lifecycle from prototyping to production. It supports local development, notebook environments, and scaled container deployments. Pathway integrates seamlessly with live data sources such as SharePoint, Google Drive, S3, Delta Tables, Kafka, and over 300 APIs, ensuring that your data remains current and synchronized. Applications can be deployed securely using Docker or Kubernetes, whether on-premises or in any cloud environment, simplifying configurations by eliminating the need for multiple databases and compute engines.
Pathway offers flexibility through ready-to-use code templates applicable to various industries and data types, which can be customized using its extensive library of functions, connectors, and integrations. This enables safe interactions with LLMs and external asynchronous APIs, thereby enhancing your data processing capabilities. By leveraging Pathway, enterprises can unlock new use cases, streamline workflows, and achieve real-time intelligence with ease.
The container uses the community version of Pathway by default. You can access the paid features by providing the license key.
Highlights
- Pathway offers a blend of batch and streaming processing capabilities, powered by a highly efficient Rust engine. This integration allows for real-time intelligence and lightning-fast data transformations, ensuring your enterprise data is always up-to-date and actionable.
- Designed for simplicity and efficiency, Pathway supports local development, notebook environments, and scalable container deployments via Docker or Kubernetes. It seamlessly connects with over 300 APIs and numerous live data sources like SharePoint, Google Drive, S3, and Kafka, providing a robust, synchronized data ecosystem.
- Pathway provides ready-to-use code templates tailored to diverse industries, which can be easily customized using its comprehensive library of functions and integrations. This flexibility enables secure interactions with LLMs and external APIs, empowering enterprises to develop and deploy advanced data processing solutions that align with their specific requirements.
Details
Sold by
Categories
Delivery method
Supported services
Delivery option
Container image
Latest version
Operating system
Linux
Deployed on AWS
Unlock automation with AI agent solutions
Fast-track AI initiatives with agents, tools, and solutions from AWS Partners.
Features and programs
Financing for AWS Marketplace purchases
AWS Marketplace now accepts line of credit payments through the PNC Vendor Finance program. This program is available to select AWS customers in the US, excluding NV, NC, ND, TN, & VT.
Pricing
Pricing and entitlements for this product are managed through an external billing relationship between you and the vendor. You activate the product by supplying a license purchased outside of AWS Marketplace, while AWS provides the infrastructure required to launch the product. AWS Subscriptions have no end date and may be canceled any time. However, the cancellation won't affect the status of the external license.
Additional AWS infrastructure costs may apply. Use the AWS Pricing Calculator to estimate your infrastructure costs.
Vendor refund policy
No refund.
Legal
Vendor terms and conditions
Upon subscribing to this product, you must acknowledge and agree to the terms and conditions outlined in the vendor's End User License Agreement (EULA)Â .
Content disclaimer
Vendors are responsible for their product descriptions and other product content. AWS does not warrant that vendors' product descriptions or other product content are accurate, complete, reliable, current, or error-free.
Delivery details
Container image
Supported services: Learn moreÂ
- Amazon EKS
- Amazon ECS
Container image
Containers are lightweight, portable execution environments that wrap server application software in a filesystem that includes everything it needs to run. Container applications run on supported container runtimes and orchestration services, such as Amazon Elastic Container Service (Amazon ECS) or Amazon Elastic Kubernetes Service (Amazon EKS). Both eliminate the need for you to install and operate your own container orchestration software by managing and scheduling containers on a scalable cluster of virtual machines.
Version release notes
Added
- path_filter parameter in pw.io.s3.read and pw.io.minio.read functions. It enables post-filtering of object paths using a wildcard pattern (*, ?), allowing exclusion of paths that pass the main path filter but do not match path_filter.
- Input connectors now support backpressure control via max_backlog_size, allowing to limit the number of read events in processing per connector. This is useful when the data source emits a large initial burst followed by smaller, incremental updates.
- pw.reducers.count_distinct and pw.reducers.count_distinct_approximate to count the number of distinct elements in a table. The pw.reducers.count_distinct_approximate allows you to save memory by decreasing the accuracy. It is possible to control this tradeoff by using the precision parameter.
- pw.Table.join (and its variants) now has two additional parameters - left_exactly_once and right_exactly_once. If the elements from a side of a join should be joined exactly once, *_exactly_once parameter of the side can be set to True. Then after getting a match an entry will be removed from the join state and the memory consumption will be reduced.
Changed
- Delta table compression logging has been improved: logs now include table names, and verbose messages have been streamlined while preserving details of important processing steps.
- Improved initialization speed of pw.io.s3.read and pw.io.minio.read.
- pw.io.s3.read and pw.io.minio.read now limit the number and the total size of objects to be predownloaded.
- BREAKING optimized the implementation of pw.reducers.min, pw.reducers.max, pw.reducers.argmin, pw.reducers.argmax, pw.reducers.any reducers for append-only tables. It is a breaking change for programs using operator persistence. The persisted state will have to be recomputed.
- BREAKING optimized the implementation of pw.reducers.sum reducer on float and np.ndarray columns. It is a breaking change for programs using operator persistence. The persisted state will have to be recomputed.
- BREAKING the implementation of data persistence has been optimized for the case of many small objects in filesystem and S3 connectors. It is a breaking change for programs using data persistence. The persisted state will have to be recomputed.
- BREAKING the data snapshot logic in persistence has been optimized for the case of big input snapshots. It is a breaking change for programs using data persistence. The persisted state will have to be recomputed.
- Improved precision of pw.reducers.sum on float columns by introducing Neumeier summation.
Additional details
Usage instructions
Run docker image
Pull the Container Image: Ensure you have pulled the container image from Amazon ECR using the provided container image details: docker pull <AWS MP Container image URL>
Launch the Docker Container: Run the following command to launch the Docker container: docker run -ti <AWS MP Container image URL>
Use the Docker Image in a Dockerfile
You can also use this Docker image in a Dockerfile to build a custom script. Here is an example Dockerfile:
FROM <AWS MP Container image URL>
COPY your_script.py /app/your_script.py
CMD ["python", "/app/your_script.py"]
Build your custom Docker image with the following command: docker build -t my-custom-image
Then, run your custom Docker image: docker run -ti my-custom-image
If you are a Pathway Scale or Pathway Enterprise user, you can provide the license key in one of the following ways:
- Set the PATHWAY_LICENSE_KEY environment variable inside the container.
- Specify the key directly as the license_key parameter in the pw.run method.
The license key can be obtained at https://pathway.com/features/Â
Resources
Vendor resources
Support
Vendor support
Support is primarily offered through Pathway's Discord channel and GitHub issues. Users can engage with the community to seek advice, report issues, and collaborate on solutions. This level of support is open and free under BSL 1.1 (Business Source License).
For all support inquiries, including community-driven discussions and issue reports, Pathway commits to providing an initial response within one business day. This ensures that users receive prompt acknowledgment and initial guidance for their queries.
Paid tier customers also have business support with tickets and 24/7 phone support depending on their plan.
AWS infrastructure support
AWS Support is a one-on-one, fast-response support channel that is staffed 24x7x365 with experienced and technical support engineers. The service helps customers of all sizes and technical abilities to successfully utilize the products and features provided by Amazon Web Services.
Similar products
Pathway
By Pathway
Hosted ELT and ETL pipelines for stream processing, real-time analytics, LLM workflows, and RAG, tailored to your needs.
Digital Care Pathway: COVID-19 Remote Monitoring (SG)
By ZEDOC by The Clinician
ZEDOC's COVID-19 Digital Care Pathway is an end-to-end solution for remote monitoring of individuals with suspected for confirmed COVID-19. Automating the collection and analysis of COVID-19 symptoms, the pathway alerts clinical teams of deterioration as it happens.
Digital Care Pathway: COVID-19 Remote Monitoring (AU)
By ZEDOC by The Clinician
ZEDOC's COVID-19 Digital Care Pathway is an end-to-end solution for remote monitoring of individuals with suspected for confirmed COVID-19. Automating the collection and analysis of COVID-19 symptoms, the pathway alerts clinical teams of deterioration as it happens.
Digital Care Pathway: COVID-19 Remote Monitoring (NZ)
By ZEDOC by The Clinician
ZEDOC's COVID-19 Digital Care Pathway is an end-to-end solution for remote monitoring of individuals with suspected for confirmed COVID-19. Automating the collection and analysis of COVID-19 symptoms, the pathway alerts clinical teams of deterioration as it happens.
Customer reviews
No customer reviews yet
Be the first to review this product . We've partnered with PeerSpot to gather customer feedback. You can share your experience by writing or recording a review, or scheduling a call with a PeerSpot analyst.