Skip to content

Commit

Permalink
QuickInstall support (#94)
Browse files Browse the repository at this point in the history
See this issue: cargo-bins/cargo-quickinstall#27

Quick Install is a hosted repo of built crates, essentially. The approach I've taken here is
a list of strategies:

1. First, we check the crate meta or default and build the URL to the repo. Once we have
   that, we perform a `HEAD` request to the URL to see if it's available.
2. If it's not, we build the URL to the quickinstall repo, and perform a `HEAD` to there.

As soon as we've got a hit, we use that. I've built it so it's extensible with more strategies.
This could be useful for #4.

This also adds a prompt before downloading from third-party sources, and logs a short
name for a source, which is easier to glance than a full URL, and includes a quick refactor
of the install/link machinery.
  • Loading branch information
passcod authored Feb 16, 2022
1 parent e691255 commit 370ae05
Show file tree
Hide file tree
Showing 12 changed files with 602 additions and 211 deletions.
13 changes: 13 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,8 @@ crates-index = "0.18.5"
semver = "1.0.5"
xz2 = "0.1.6"
zip = "0.5.13"
async-trait = "0.1.52"
url = "2.2.2"

[dev-dependencies]
env_logger = "0.9.0"
8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ yes
- [x] Fetch crate / manifest via crates.io
- [ ] Fetch crate / manifest via git (/ github / gitlab)
- [x] Use local crate / manifest (`--manifest-path`)
- [x] Fetch build from the [quickinstall](https://github.com/alsuren/cargo-quickinstall) repository
- [ ] Unofficial packaging
- Package formats
- [x] Tgz
Expand Down Expand Up @@ -126,6 +127,10 @@ By default `binstall` is setup to work with github releases, and expects to find

If your package already uses this approach, you shouldn't need to set anything.

### QuickInstall

[QuickInstall](https://github.com/alsuren/cargo-quickinstall) is an unofficial repository of prebuilt binaries for Crates, and `binstall` has built-in support for it! If your crate is built by QuickInstall, it will already work with `binstall`. However, binaries as configured above take precedence when they exist.

### Examples

For example, the default configuration (as shown above) for a crate called `radio-sx128x` (version: `v0.14.1-alpha.5` on x86_64 linux) would be interpolated to:
Expand Down Expand Up @@ -166,9 +171,6 @@ Which provides a binary path of: `sx128x-util-x86_64-unknown-linux-gnu[.exe]`. I
- Why use the cargo manifest?
- Crates already have these, and they already contain a significant portion of the required information.
Also there's this great and woefully underused (imo) `[package.metadata]` field.
- Why not use a binary repository instead?
- Then we'd need to _host_ a binary repository, and worry about publishing and all the other fun things that come with releasing software.
This way we can use existing CI infrastructure and build artifacts, and maintainers can choose how to distribute their packages.
- Is this secure?
- Yes and also no? We're not (yet? #1) doing anything to verify the CI binaries are produced by the right person / organisation.
However, we're pulling data from crates.io and the cargo manifest, both of which are _already_ trusted entities, and this is
Expand Down
1 change: 0 additions & 1 deletion build.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@

// Fetch build target and define this for the compiler
fn main() {
println!(
Expand Down
125 changes: 125 additions & 0 deletions src/bins.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
use std::path::PathBuf;

use cargo_toml::Product;
use serde::Serialize;

use crate::{PkgFmt, PkgMeta, Template};

pub struct BinFile {
pub base_name: String,
pub source: PathBuf,
pub dest: PathBuf,
pub link: PathBuf,
}

impl BinFile {
pub fn from_product(data: &Data, product: &Product) -> Result<Self, anyhow::Error> {
let base_name = product.name.clone().unwrap();

// Generate binary path via interpolation
let ctx = Context {
name: &data.name,
repo: data.repo.as_ref().map(|s| &s[..]),
target: &data.target,
version: &data.version,
format: if data.target.contains("windows") {
".exe"
} else {
""
},
bin: &base_name,
};

// Generate install paths
// Source path is the download dir + the generated binary path
let source_file_path = ctx.render(&data.meta.bin_dir)?;
let source = if data.meta.pkg_fmt == PkgFmt::Bin {
data.bin_path.clone()
} else {
data.bin_path.join(&source_file_path)
};

// Destination path is the install dir + base-name-version{.format}
let dest_file_path = ctx.render("{ bin }-v{ version }{ format }")?;
let dest = data.install_path.join(dest_file_path);

// Link at install dir + base name
let link = data.install_path.join(&base_name);

Ok(Self {
base_name,
source,
dest,
link,
})
}

pub fn preview_bin(&self) -> String {
format!(
"{} ({} -> {})",
self.base_name,
self.source.file_name().unwrap().to_string_lossy(),
self.dest.display()
)
}

pub fn preview_link(&self) -> String {
format!(
"{} ({} -> {})",
self.base_name,
self.dest.display(),
self.link.display()
)
}

pub fn install_bin(&self) -> Result<(), anyhow::Error> {
// TODO: check if file already exists
std::fs::copy(&self.source, &self.dest)?;

#[cfg(target_family = "unix")]
{
use std::os::unix::fs::PermissionsExt;
std::fs::set_permissions(&self.dest, std::fs::Permissions::from_mode(0o755))?;
}

Ok(())
}

pub fn install_link(&self) -> Result<(), anyhow::Error> {
// Remove existing symlink
// TODO: check if existing symlink is correct
if self.link.exists() {
std::fs::remove_file(&self.link)?;
}

#[cfg(target_family = "unix")]
std::os::unix::fs::symlink(&self.dest, &self.link)?;
#[cfg(target_family = "windows")]
std::os::windows::fs::symlink_file(&self.dest, &self.link)?;

Ok(())
}
}

/// Data required to get bin paths
pub struct Data {
pub name: String,
pub target: String,
pub version: String,
pub repo: Option<String>,
pub meta: PkgMeta,
pub bin_path: PathBuf,
pub install_path: PathBuf,
}

#[derive(Clone, Debug, Serialize)]
struct Context<'c> {
pub name: &'c str,
pub repo: Option<&'c str>,
pub target: &'c str,
pub version: &'c str,
pub format: &'c str,
pub bin: &'c str,
}

impl<'c> Template for Context<'c> {}
101 changes: 62 additions & 39 deletions src/drivers.rs
Original file line number Diff line number Diff line change
@@ -1,39 +1,43 @@

use std::time::Duration;
use std::path::{Path, PathBuf};
use std::time::Duration;

use log::{debug};
use anyhow::{Context, anyhow};
use anyhow::{anyhow, Context};
use log::debug;
use semver::{Version, VersionReq};

use crates_io_api::AsyncClient;

use crate::PkgFmt;
use crate::helpers::*;
use crate::PkgFmt;

fn find_version<'a, V: Iterator<Item=&'a str>>(requirement: &str, version_iter: V) -> Result<String, anyhow::Error> {
fn find_version<'a, V: Iterator<Item = &'a str>>(
requirement: &str,
version_iter: V,
) -> Result<String, anyhow::Error> {
// Parse version requirement
let version_req = VersionReq::parse(requirement)?;

// Filter for matching versions
let mut filtered: Vec<_> = version_iter.filter(|v| {
// Remove leading `v` for git tags
let ver_str = match v.strip_prefix("s") {
Some(v) => v,
None => v,
};

// Parse out version
let ver = match Version::parse(ver_str) {
Ok(sv) => sv,
Err(_) => return false,
};

debug!("Version: {:?}", ver);

// Filter by version match
version_req.matches(&ver)
}).collect();
let mut filtered: Vec<_> = version_iter
.filter(|v| {
// Remove leading `v` for git tags
let ver_str = match v.strip_prefix("s") {
Some(v) => v,
None => v,
};

// Parse out version
let ver = match Version::parse(ver_str) {
Ok(sv) => sv,
Err(_) => return false,
};

debug!("Version: {:?}", ver);

// Filter by version match
version_req.matches(&ver)
})
.collect();

// Sort by highest matching version
filtered.sort_by(|a, b| {
Expand All @@ -48,13 +52,19 @@ fn find_version<'a, V: Iterator<Item=&'a str>>(requirement: &str, version_iter:
// Return highest version
match filtered.get(0) {
Some(v) => Ok(v.to_string()),
None => Err(anyhow!("No matching version for requirement: '{}'", version_req))
None => Err(anyhow!(
"No matching version for requirement: '{}'",
version_req
)),
}
}

/// Fetch a crate by name and version from crates.io
pub async fn fetch_crate_cratesio(name: &str, version_req: &str, temp_dir: &Path) -> Result<PathBuf, anyhow::Error> {

pub async fn fetch_crate_cratesio(
name: &str,
version_req: &str,
temp_dir: &Path,
) -> Result<PathBuf, anyhow::Error> {
// Fetch / update index
debug!("Updating crates.io index");
let mut index = crates_index::Index::new_cargo_default()?;
Expand All @@ -65,37 +75,48 @@ pub async fn fetch_crate_cratesio(name: &str, version_req: &str, temp_dir: &Path
let base_info = match index.crate_(name) {
Some(i) => i,
None => {
return Err(anyhow::anyhow!("Error fetching information for crate {}", name));
return Err(anyhow::anyhow!(
"Error fetching information for crate {}",
name
));
}
};

// Locate matching version
let version_iter = base_info.versions().iter().map(|v| v.version() );
let version_iter = base_info.versions().iter().map(|v| v.version());
let version_name = find_version(version_req, version_iter)?;

// Build crates.io api client
let api_client = AsyncClient::new("cargo-binstall (https://github.com/ryankurte/cargo-binstall)", Duration::from_millis(100))?;
let api_client = AsyncClient::new(
"cargo-binstall (https://github.com/ryankurte/cargo-binstall)",
Duration::from_millis(100),
)?;

// Fetch online crate information
let crate_info = api_client.get_crate(name.as_ref()).await
let crate_info = api_client
.get_crate(name.as_ref())
.await
.context("Error fetching crate information")?;

// Fetch information for the filtered version
let version = match crate_info.versions.iter().find(|v| v.num == version_name) {
Some(v) => v,
None => {
return Err(anyhow::anyhow!("No information found for crate: '{}' version: '{}'",
name, version_name));
return Err(anyhow::anyhow!(
"No information found for crate: '{}' version: '{}'",
name,
version_name
));
}
};

debug!("Found information for crate version: '{}'", version.num);

// Download crate to temporary dir (crates.io or git?)
let crate_url = format!("https://crates.io/{}", version.dl_path);
let tgz_path = temp_dir.join(format!("{}.tgz", name));

debug!("Fetching crate from: {}", crate_url);
debug!("Fetching crate from: {}", crate_url);

// Download crate
download(&crate_url, &tgz_path).await?;
Expand All @@ -111,8 +132,10 @@ pub async fn fetch_crate_cratesio(name: &str, version_req: &str, temp_dir: &Path

/// Fetch a crate by name and version from github
/// TODO: implement this
pub async fn fetch_crate_gh_releases(_name: &str, _version: Option<&str>, _temp_dir: &Path) -> Result<PathBuf, anyhow::Error> {

pub async fn fetch_crate_gh_releases(
_name: &str,
_version: Option<&str>,
_temp_dir: &Path,
) -> Result<PathBuf, anyhow::Error> {
unimplemented!();
}

Loading

0 comments on commit 370ae05

Please sign in to comment.