Is there any library for the queueing mechanism?

What’s used by the most - Cron? But a task or rather script executed by Cron won’t access to the context of an application. Meaning, a task will have be an independent unit. Whereas I want is a library to use inside a project such that it’ll have access to everything.

Anything similar to Sidekiq exist in Rust?

  • ikornaselur@lemm.ee
    link
    fedilink
    arrow-up
    1
    ·
    1 year ago

    I looked into this last week for recurring tasks in an actix-web project and ended up picking fang, but haven’t really battle tested it… But seems to do what I need. I’m using cron tasks with Postgres as the backend to run a recurring task every minute.

  • vaalla@discuss.tchncs.de
    link
    fedilink
    arrow-up
    1
    ·
    1 year ago

    Can you use async in your project?

    If Yes, you can spawn a task that will listen on a channel. If you need to run them in parallel probably you can find a mpmc channel.

    If you need for them to run at a specific time, spawn task ,tokio::time::sleep , run job, loop.

    Don’t know any crate just for this.

    • nothingness@lemmy.worldOP
      link
      fedilink
      arrow-up
      1
      arrow-down
      1
      ·
      1 year ago

      If you need for them to run at a specific time, spawn task ,tokio::time::sleep , run job, loop.

      How would you do it every 30 minutes? Every 5 hours? Once a day?

  • kevincox@lemmy.ml
    link
    fedilink
    arrow-up
    1
    ·
    edit-2
    1 year ago

    For my project I just run them. I’m using async so it is just spawning a task in the background, but you can do the same with threads. Don’t underestimate the number of threads that you can run on a modern computer.

    If you want some sort of throttling you can stuff tasks into a queue and just run N background threads pulling them off an processing them.

    If you need durability then you start to have more trouble. This is where I would start looking at a library. IDK if there are any libraries that handle logging, retries and similar, but if not you can probably get the basics down pretty easily.

    • nothingness@lemmy.worldOP
      link
      fedilink
      arrow-up
      1
      arrow-down
      1
      ·
      edit-2
      1 year ago

      For my project I just run them.

      How would you “just run” a task every 30 minutes? Every 5 hours? Once a day?

      • kevincox@lemmy.ml
        link
        fedilink
        arrow-up
        2
        ·
        1 year ago
        std::thread::spawn(|| {
            loop {
                std::thread::sleep(std::time::Duration::from_secs(30*60));
                do_job();
            }
        });
        

        Works pretty well. Maybe add a bit of code to crash the whole process on panic or some other logging. Wastes a few KiB of memory per loop but probably not a major issue. Doing this with async will waste only the tiniest amount of memory.

          • kevincox@lemmy.ml
            link
            fedilink
            arrow-up
            1
            ·
            1 year ago

            It depends. Sometimes you can just put an exit call at the end of main to kill the thread. If you want to attempt graceful shutdown then usually I just use a boolean shutdown flag. Then the loop becomes while !shutdown.get() {

          • kevincox@lemmy.ml
            link
            fedilink
            arrow-up
            1
            ·
            1 year ago

            Best option is probably to add a wrapper around the thread that re-spawns it. But you can also just catch panics in the loop.

            • nothingness@lemmy.worldOP
              link
              fedilink
              arrow-up
              1
              arrow-down
              1
              ·
              edit-2
              1 year ago

              How would you re-run it multiple times then? An internal should be progressively greater. How would you terminate it if it continues to produce an exception?