GitHub shouldn't be a dependency for publishing Rust on crates.io

(infosec.exchange)

133 points | by speckx 4 hours ago

13 comments

  • epage 3 hours ago
    An RFC was recently merged to unblock this: https://github.com/rust-lang/rfcs/pull/3963

    The implementation on this has started.

    Something to keep in mind is https://blog.m-ou.se/rust-is-not-a-company/. Rust is mostly driven by volunteers working on what they find interesting. Boring/uninteresting tasks depend on funding, a warm body to accept the funding, and a reviewer.

    • echelon 2 hours ago
      It's not just that this is boring work, but there's disagreement about Cargo and crates.io's direction. There are a lot of changes people would like to make that get turned down.

      Crates.io and Cargo need namespaces, but the leadership flatly says no.

      There's a big problem with name squatting, and nothing is being done about this either.

      I get that there are more technically important issues around builds and reproducibility and the like, but this is pretty foundational stuff.

      • epage 14 minutes ago
        That and the existing reply are incorrect. Almost everyone has repeated the same superficial requests and design work and not dug into the problems to come up with a proposal that respectfully threads seemingly contradictory requirements (the answer isn't to be dismissive but to step through costs/benefits and explain why a path is justified).

        https://internals.rust-lang.org/t/survey-of-organizational-o... is a start in just collecting existing knowledge in one place.

      • stymaar 2 hours ago
        > Crates.io and Cargo need namespaces, but the leadership flatly says no.

        They are favorable to crate-name-as-namespace (so that once you have the tokio crate you can use tokio as a namespace) and there's ongoing work on that. But as said above, it takes work to implement.

        There's no desire for other meaning of the word "namespace" because famously nobody ever made a well-reasoned proposal (despite the amount of social media outrage over the lack of namespace).

  • vsgherzi 1 hour ago
    I agree and so does the rust project. The main problem is that it's alot of work and it's hard.

    https://www.youtube.com/watch?v=zGS-HqcAvA4 Here's a long video from jon gjengset that shows how it works and some of the effort already done to de-couple from github.

    Crates is widely used so it's a rebuilding the track while the train is driving kind of problem.

    It's just not a priority for the project right now, but I would also definitely like to see the issue done. It might be good for the rust project to vote on things like this during surveys so they know where to focus work!

  • r2vcap 19 minutes ago
    From a supply-chain perspective, Cargo is still in the same broad risk category as npm and PyPI: installing packages means trusting externally published code, including code that may execute during build or installation.

    Rather than looking for someone to blame - in this case, GitHub - we should focus on constructive ways to harden the ecosystem.

  • ameliaquining 3 hours ago
    See the official project issue on this: https://github.com/rust-lang/crates.io/issues/326

    TL;DR: They want to fix this, it's a lot of work that no one's being paid to do, there's a roadmap with specific tasks that need doing, volunteer contributions are welcome.

    • DyslexicAtheist 3 hours ago
      > it's a lot of work that no one's being paid to do,

      aren't they like some kind of non-profit (in the legal sense) that is still able to take a lot of money (from players like Google and Co, to justify fixing this), as opposed to ... say the Zig foundation, ... that is is also "non-profit" but can't get money the same way?

      • jojomodding 2 hours ago
        The non-profit (the Foundation) pays for specific things but it is not really there to hire people to work on things. It pays for infrastructure work and to pay the existing maintainers who often do review work. It also gives stipends to up-and-coming contributors for Open Source outreach programmes, but this are not really the people who you want to have immediately work on your critical infrastructure code.
    • sscaryterry 3 hours ago
      Just going to say it out loud :) Its been known for 10 years.

      https://en.wikipedia.org/wiki/The_purpose_of_a_system_is_wha...

      • NobodyNada 3 hours ago
        10 years ago, GitHub had a far better reputation and the Rust ecosystem was much smaller and less load-bearing, so "what if someone doesn't have a GitHub account" was a theoretical concern for most people. So the issue was a low-priority backlog item that everyone agreed would be nice-to-have but there weren't enough people willing to volunteer their time to it over more important and more impactful work.

        Obviously, the situation has changed in recent years, so it's now considered a much higher priority by many people and some of them are actively working on it. But it's a lot of work to be done by volunteers, so it takes time.

        That's the reality of open-source projects: things get done when they are important enough to motivate someone to either fund it or work on in their free time, not according to idyllic roadmaps and schedules.

        • dijit 2 hours ago
          The reason people were sounding the alarm 10 years ago is because if you tie yourself to a proprietary platform then you're at its mercy, even if it changes for the worse for everyone which is what we're seeing now.
          • arjie 1 hour ago
            With open-source projects, realistically there is no shortage of alarm sounding, and there is a shortage of alarm fixing, consequently if you really care about this being fixed you have to volunteer to go fix it.
          • janalsncm 2 hours ago
            The comment you are replying to was in response to essentially the same point, albeit with fewer words and less emphasis.
          • kibwen 1 hour ago
            From the outset, crates.io was careful to deliberately not tie itself inextricably to Github. For example, by resisting the endless deluge of people demanding that Github usernames be used as public-facing package namespaces. Github is only used as an identity provider for logging in.
        • sscaryterry 3 hours ago
          Pro tip: Using "load-bearing" is heavily associated with LLM usage :)
        • sscaryterry 3 hours ago
          Wow, have you forgotten? https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguis...

          10 (edit: 8) years ago MS took over Github. The writing was on the wall then...

          No need to explain OSS to me, I maintain and contribute.

      • ameliaquining 2 hours ago
  • mikey_p 2 hours ago
    The longer I go the more I have actually come to appreciate the way Packagist works for the PHP community, there are lots of cool things it does that I wish NPM or other registries did by default, like forcing you to package from a source repository, so that you can't upload a different artifact from what you keep in source control.
    • ecshafer 2 hours ago
      How does a close source package work? Depending on the language its not super helpful, but a package that is closed source should be possible.
      • hmry 1 hour ago
        For crates.io: They don't allow closed-source packages. But they're just the free community package index, you're not forced to use them.

        You can:

        - host a private index

        - host the proprietary binaries in a git repo and use a git dependency

        - commit the proprietary binaries into your source repo, and use a path dependency

    • steveklabnik 1 hour ago
      crates.io also only distributes source code.
  • Animats 3 hours ago
    Sadly, that's probably correct. No outside single point of failure that can cancel users at will can be allowed to gatekeep open source projects.
  • sscaryterry 3 hours ago
    Especially not now, what if they're down? ;)
    • veqq 2 hours ago
      This is a big issue. https://janetdocs.org/ handles auth through GH which leads to... regular problems, unfortunately. I hope to migrate soon.
  • dboreham 1 hour ago
    My take: publishing Rust crates shouldn't depend on any single internet property, including crates.io.
    • deathanatos 16 minutes ago
      I think crates.io is essentially just the default, and you can point cargo to an alternate package repository, if you so desire.

      I've worked on projects where we vendored all third-party crates, for example, so our config just pointed to that vendoring, and I think support ought to be better these days…

  • lorecore 1 hour ago
    Go handles this well, kind of. It's super easy (in fact, transparent) to import from GitHub urls. You can self-host your Go packages, but it involves making and hosting some manifest files. Not as seamless as using GitHub, but still totally doable.
    • fyrn_ 19 minutes ago
      In practice, Go is much, much more github dependent than rust, most go packages are on github
  • rho138 48 minutes ago
    Holy fuck it’s been a decade. Nothing is that complex.
    • adamch 6 minutes ago
      PRs are welcome.
  • jauntywundrkind 2 hours ago
    The teams support may be a bit trickier/less clear to move on, but generally: this feels like a great place where atproto / bluesky support would slot in well.
    • dnfmfnfnfb 1 hour ago
      I don’t see how that would be effectively useful for any aspect of the problems
      • steveklabnik 1 hour ago
        The only thing GitHub is used as on crates.io is as an identity provider. Using your atproto identity is pretty straightforwardly a conceptual substitute.
  • androiddrew 2 hours ago
    Welcome to Golang packaging problems. Hope you get it sorted out
    • sshine 2 hours ago
      But Sylvain Kerkour says Go's approach is much better than Rust's!
    • steveklabnik 1 hour ago
      The shape is very different. The only thing crates.io uses GitHub for is for identity.
    • bsder 1 hour ago
      Can someone explain to me why the inverse domain name solution that everyone in the Java world converged on doesn't work?

      It's really not clear to me why people keep avoiding that.

  • righthand 3 hours ago
    Aka one of the many Rust reasons why I chose to learn C.
    • hmry 2 hours ago
      Using crates.io is entirely optional, you can download a library's source code and specify the path to it in your cargo config file. (Which is not uncommon in production)

      For that matter, using cargo is optional, you can compile rust code using GNU make or shell scripts if you want to. (That's what the Linux kernel does)

    • antonvs 1 hour ago
      Welcome to 1972, young padawan. You have a long journey ahead of you.