Google Cloud: Why we migrated from PubSub to Cloud Tasks

When working with Google Cloud Platform (GCP), two of the many services you might encounter for message delivery and asynchronous processing are Cloud Tasks and Pub/Sub. Both tools are designed to ensure that applications can be decoupled, scalable, and reliable. However, the use cases for each service differ. Here, we dive deep into these two services and share why a seemingly small feature, the retry & delivery rate settings, prompted us to switch from PubSub to Cloud Tasks.

Cloud Tasks: At a Glance

Cloud Tasks is a fully managed service that allows for asynchronous execution of tasks. It can be seen as a task queue with the ability to schedule, manage, and track billions of tasks at once.

Key Features

1. Delayed Execution: Schedule tasks to be run at specific times or after a specific delay.
2. Rate Limiting: Control the rate at which tasks are processed to manage the load on services.
3. De-duplication: Avoid executing duplicate tasks. Retry Mechanism: If a task fails, Cloud Tasks can retry it based on configurable policies.

Use Cases

• Backend processing, like sending emails or processing uploaded data.
• Deferring tasks that do not need to be executed immediately.
• Executing heavy computations outside of the main user flow to ensure responsiveness.

Pub/Sub: At a Glance

Google Cloud Pub/Sub provides a messaging service that allows you to decouple services that produce events (publishers) from services that process events (subscribers). It is a distributed messaging service that can scale as your data grows.

Key Features

1. Real-time Messaging: Supports real-time messaging between independent applications.
2. At-Least-Once Delivery: Messages are delivered at least once, ensuring that no messages are missed.
3. Global Distribution: Messages can be sent and received globally.

Use Cases

• Streaming data for real-time analytics.
• Integrating disparate systems in a microservices architecture.
• Distributing event notifications.
• Processing and moving data between different parts of a system in real-time.

Cloud Tasks vs Pub/Sub: The Showdown

While both services offer asynchronous processing, they are tailored for different scenarios:

1. Granularity: Cloud Tasks is more about individual task management. It's perfect when you need precise control over when and how tasks are executed. On the other hand, Pub/Sub is all about streaming vast amounts of data and messages between systems.
2. Reliability: Both systems provide mechanisms to ensure that tasks/messages are processed but approach them differently. Cloud Tasks focuses on retries for individual tasks, while Pub/Sub ensures that messages are delivered at least once to subscribers.
3. Scope: If you're looking to handle event-driven architectures, especially with multiple subscribers, Pub/Sub is the way to go. However, Cloud Tasks might be more appropriate for backend processing tasks that aren’t event-driven.
4. Integration: Cloud Tasks is tightly integrated with App Engine and can call HTTP endpoints directly. Pub/Sub, while versatile, might require more setup for similar integrations.

Migrating from Pub/Sub to Cloud Tasks

Our story is as much about growth as it is about technological requirements. As our application scaled, the volume of messages we dealt with on PubSub grew exponentially. It soon became evident that PubSub's retry mechanism was causing more chaos than convenience.

Here's why:

• Delivery Rate: PubSub’s push subscription sends messages as soon as they are available, a behaviour we found overwhelming our endpoints at times. This lack of control could potentially overload servers, especially during traffic surges. Contrastingly, Cloud Tasks offers rate configurations, preventing such overloads by controlling the rate at which tasks are dispatched.
• Retry Mechanism: We realised that PubSub's retry mechanism didn't offer us the granularity we needed to handle failures differently based on their cause. Some of the failed tasks should be retried after a considerable duration, sometimes well beyond the 10-minute mark that PubSub restricts. Cloud Tasks allowed us to extend this duration, providing the flexibility we sought.
• Operational Overheads: Our teams spent an inordinate amount of time identifying genuine failures from those simply awaiting another retry.

With these challenges at hand, we began looking at alternatives and soon, Cloud Tasks emerged as the ideal solution.

The Migration Journey

First, we initiated by analysing the kinds of tasks that demanded a more controlled retry mechanism. This gave us a blueprint of what our Cloud Tasks setup should look like. We then established our Cloud Task queues and handlers with tailored retry configurations for various task categories. Instead of an immediate switch, we ran Cloud Tasks alongside PubSub, starting with tasks that faced the most challenges under PubSub’s retry mechanism. Once confident in Cloud Tasks' robustness, we rerouted all tasks from PubSub, concluding our transition.

The Aftermath: Benefits of the Switch

• Granular Control: Cloud Tasks' retry configurations allowed us to manage retries with unprecedented precision.
• Operational Peace: Our teams could now focus on genuine issues instead of getting bogged down by retry noise.

Conclusion

While both Cloud Tasks and PubSub offer immense value, detailed features like retry settings can dictate which is more apt for your scenario. Our migration story underscores the importance of aligning technology choices with evolving operational requirements. Ensure you weigh the details when choosing; they often make all the difference.