Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix tests by making tests a package instead of an application #1104

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions tests/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
.elm
32 changes: 12 additions & 20 deletions tests/elm.json
Original file line number Diff line number Diff line change
@@ -1,26 +1,18 @@
{
"type": "application",
"source-directories": [],
"elm-version": "0.19.1",
"type": "package",
"name": "elm/core-tests",
"summary": "Elm's standard libraries",
"license": "BSD-3-Clause",
"version": "1.0.5",
"elm-version": "0.19.0 <= v < 0.20.0",
"exposed-modules": [
"EmptyPlaceholderModule"
],
"dependencies": {
"direct": {
"elm/browser": "1.0.1",
"elm/core": "1.0.2",
"elm/html": "1.0.0"
},
"indirect": {
"elm/json": "1.1.2",
"elm/time": "1.0.0",
"elm/url": "1.0.0",
"elm/virtual-dom": "1.0.2"
}
"elm/core": "1.0.0 <= v < 2.0.0"
},
"test-dependencies": {
"direct": {
"elm-explorations/test": "1.2.1"
},
"indirect": {
"elm/random": "1.0.0"
}
"elm/json": "1.1.3 <= v < 2.0.0",
"elm-explorations/test": "1.2.2 <= v < 2.0.0"
}
}
49 changes: 11 additions & 38 deletions tests/run-tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,68 +3,41 @@
set -o errexit;
set -o nounset;

#let the caller supply an ELM_TEST binary if desired
# Let the caller supply an ELM_TEST binary if desired.
if [ -z "${ELM_TEST:-}" ]; then
ELM_TEST=elm-test;
fi

# since elm/core is treated specially by the compiler (it's always
# Since elm/core is treated specially by the compiler (it's always
# inserted as a dependency even when not declared explicitly), we use
# a bit of a hack to make the tests run against the local source code
# rather than the elm/core source fetched from package.elm-lang.org.

# create a local directory where the compiler will look for the
# Create a local directory where the compiler will look for the
# elm/core source code:

DIR="$(dirname $0)";

cd "$DIR";

export ELM_HOME="$(pwd)/.elm";

export ELM_HOME="$PWD/.elm";
rm -rf "$ELM_HOME" && mkdir -p "$ELM_HOME";

# elm-test also puts some things in elm-stuff, start with a clean
# slate there as well

rm -rf elm-stuff;

# now make an initial run of the tests to populate .elm and elm-stuff;
# this will test against elm/core from package.elm-lang.org, so we
# don't really care what the results are; we just need to force all
# the *other* dependencies to be fetched and set up.

echo "seeding framework for test dependencies ...";

# '|| true' lets us ignore failures here and keep the script running.
# useful when developing a fix for a bug that exists in the version of
# elm/core hosted on package.elm-lang.org
"${ELM_TEST}" tests/Main.elm --fuzz=1 > /dev/null || true;

# clear out the copy of elm-core fetched by the above and replace it
# with the local source code we want to actually test

VERSION_DIR="$(ls ${ELM_HOME}/0.19.1/packages/elm/core/)"
CORE_PACKAGE_DIR="${ELM_HOME}/0.19.1/packages/elm/core/$VERSION_DIR"
# Create a link to the git package
CORE_LINK="${ELM_HOME}/0.19.1/packages/elm/core/1.0.5"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You might want to add a comment here saying that setting this to 1.0.5 is fine as long as the strategy used by the dependency solver in elm-test (here elm-json solve) prioritizes installed versions. In such case, it will use the linked 1.0.5 to the git repo even when a new 1.0.6 version of core exists. This of course only holds true as long as the build-time and run-time tests dependencies do not require a version of elm/core > 1.0.5.

Otherwise, we need to bump this 1.0.5 once there is a new release of elm/core.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about this: a888bfe hopefully this will keep us in sync regardless of the dependency resolution

Copy link

@mpizenberg mpizenberg Apr 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose this does the trick in 99% of the cases. The only exception I can think of is for the commit that changes the package version. It will try to use the new version that does not exist in the package server yet and thus will make the elm compiler fail I think. Might be annoying if this shows a red failure for CI on the commit corresponding to new package push.
What do you think?

PS: except if your elm publish is faster than your CI but that's a bit playing the daredevil ^^

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, this is a problem. Maybe we need to go back to the old approach of running elm-test once to prepopulate elm_home and then simlink

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well you could also just ask the package server which version of elm-core is the latest.

Copy link

@mpizenberg mpizenberg Apr 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

edited

curl -L https://package.elm-lang.org/packages/elm/core/releases.json | jq -r 'keys | .[-1]'
# 1.0.5

CORE_GIT_DIR="$(dirname $PWD)"

echo;
echo "Linking $CORE_PACKAGE_DIR to $CORE_GIT_DIR"
echo "Linking $CORE_LINK to $CORE_GIT_DIR"
echo;

rm -rf "$CORE_PACKAGE_DIR"
ln -sv "$CORE_GIT_DIR" "$CORE_PACKAGE_DIR"
mkdir -p "$(dirname $CORE_LINK)"
ln -sv "${CORE_GIT_DIR}" "${CORE_LINK}"
rm -vf "${CORE_GIT_DIR}"/*.dat "${CORE_GIT_DIR}"/doc*.json

# we also need to clear out elm-test's elm-stuff dir, since otherwise
# the compiler complains that its .dat files are out of sync

rm -rf elm-stuff;

# now we can run the tests against the symlinked source code for real
# Now we can run the tests against the symlinked source code for real.

echo;
echo "running tests ...";
echo;

"${ELM_TEST}" tests/Main.elm "$@";
"${ELM_TEST}" "$@";
Empty file added tests/src/empty
Empty file.
40 changes: 0 additions & 40 deletions tests/tests/Main.elm

This file was deleted.