Skip to content

Latest commit

 

History

History
468 lines (364 loc) · 27.2 KB

SETUP.md

File metadata and controls

468 lines (364 loc) · 27.2 KB

Setup

This document describes how to set up your workstation to develop for Code.org.

You can do Code.org development using OSX, Ubuntu, or Windows (running Ubuntu in a VM). Setup for Windows is more complicated and relatively few developers use it. Make sure you follow the instructions for your platform in the subsections below.

Overview

  1. Install OS-specific prerequisites

    • See the appropriate section below: OS X, Ubuntu, Windows

    • Important: When done, check for correct versions of these dependencies:

      ruby --version  # --> ruby 2.5.0
      node --version  # --> v8.15.0
      yarn --version  # --> 1.16.0
      
  2. If using SSH (recommended): git clone [email protected]:code-dot-org/code-dot-org.git , if using HTTPS: git clone https://github.com/code-dot-org/code-dot-org.git,

  3. gem install bundler -v 1.17.3

  4. rbenv rehash

  5. cd code-dot-org

  6. bundle install

    • This step often fails to due environment-specific issues. Look in the Bundle Install Tips section below for steps to resolve many common issues.
  7. bundle exec rake install:hooks

    Troubleshoot: `rake aborted! Gem::LoadError: You have already activated...`
    • If you have issue "rake aborted! Gem::LoadError: You have already activated rake 12.3.0, but your Gemfile requires rake 11.3.0.", make sure you add bundle exec in front of the rake install:hooks command
    Troubleshoot: wrong version of rake
    • You might get a message at some point about having the wrong version of rake. If so, try:
      $> gem uninstall rake
      $> bundle update rake
      
  8. bundle exec rake install

    • This can take a long time, ~30 minutes or more. The most expensive are the "seeding" tasks, where your local DB is populated from data in the repository. Some of the seeding rake tasks can take several minutes. The longest one, seed:scripts, can take > 10 minutes, but it should at least print out progress as it goes.
  9. fix your database charset and collation to match our servers

    • bin/dashboard-sql
    • ALTER DATABASE dashboard_development CHARACTER SET utf8 COLLATE utf8_unicode_ci;
    • ALTER DATABASE dashboard_test CHARACTER SET utf8 COLLATE utf8_unicode_ci;
  10. bundle exec rake build

    • This may fail if your are on a Mac and your OSX XCode Command Line Tools were not installed properly. See Bundle Install Tips for more information.
  11. (Optional, Code.org engineers only) Setup AWS - Ask a Code.org engineer how to complete this step

    1. Some functionality will not work on your local site without this, for example, some project-backed level types such as https://studio.code.org/projects/gamelab. This setup is only available to Code.org engineers for now, but it is recommended for Code.org engineers.
  12. Run the website bin/dashboard-server

  13. Visit http://localhost-studio.code.org:3000/ to verify it is running.

  14. Install necessary plugins described in the Editor configuration section below.

After setup, read about our code styleguide, our test suites, or find more docs on the wiki.

OS-specific prerequisites

OS X Catalina

  1. Choose shell. Starting in Catalina, the default shell for new users is zsh. Most developers at Code.org are still using bash so that may be a smoother experience for now.

    To use bash:
    • Switch the default shell back to bash and disable the warning:
      • chsh -s /bin/bash
      • Add the following to ~/.bash_profile or your desired shell configuration file:
        export BASH_SILENCE_DEPRECATION_WARNING=1
        
    Optional configuration steps for zsh:
    • Setup git prompt and git autocompletion
      • Download git-prompt.sh
        mkdir -p ~/bin/oh-my-zsh
        curl https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/plugins/gitfast/git-prompt.sh > ~/bin/oh-my-zsh/git-prompt.sh
        
      • Add the following to ~/.zshrc or your desired shell configuration file:
        # git prompt
        source ~/bin/oh-my-zsh/git-prompt.sh
        GIT_PS1_SHOWCOLORHINTS=1
        GIT_PS1_SHOWDIRTYSTATE=1
        GIT_PS1_SHOWUNTRACKEDFILES=1
        setopt PROMPT_SUBST ; PS1='%m:%~$(__git_ps1 " (%s)")\$ '
         
        # git completion
        source /Library/Developer/CommandLineTools/usr/share/git-core/git-completion.zsh >/dev/null 2>&1
         
        # make git checkout not show remote branches
        GIT_COMPLETION_CHECKOUT_NO_GUESS=1
        autoload -Uz compinit && compinit
        
      • fix any problems with compinit:
        compaudit | xargs chmod g-w
        
  2. Install Homebrew: /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"

  3. Install Redis: brew install redis

  4. Run brew install https://raw.github.com/quantiverge/homebrew-binary/pdftk/pdftk.rb enscript gs [email protected] nvm imagemagick rbenv ruby-build coreutils sqlite parallel

    Troubleshoot: pdftk errors
    Troubleshoot: old version of <package>
    • If it complains about an old version of <package>, run brew unlink <package> and run brew install <package> again
  5. Set up MySQL

    1. Force link 5.7 version: brew link [email protected] --force
    2. Have launchd start mysql at login: ln -sfv /usr/local/opt/mysql/*.plist ~/Library/LaunchAgents
      1. Note, if mysql folder name is "[email protected]", replace the command above by ln -sfv /usr/local/opt/[email protected]/*.plist ~/Library/LaunchAgents. (Use ls -d /usr/local/opt/mysql* to check for folder name.)
    3. Start mysql now: launchctl load ~/Library/LaunchAgents/homebrew.mxcl.mysql.plist
      1. Note: if this fails check your plist file (ls ~/Library/LaunchAgents/) to see if it is "[email protected]". If it is try: launchctl load ~/Library/LaunchAgents/[email protected] instead
  6. Set up rbenv

    1. Run rbenv init
    2. Add the following to ~/.bash_profile or your desired shell: eval "$(rbenv init -)". More info here.
    3. Pick up those changes: source ~/.bash_profile
  7. Install Ruby 2.5.0

    1. rbenv install 2.5.0
    2. Set the global version of Ruby: rbenv global 2.5.0
    3. Install shims for all Ruby executables: rbenv rehash. More info here.
  8. Set up nvm

    1. Create nvm's working directory if it doesnt exist: mkdir ~/.nvm

    2. Add the following to ~/.bash_profile or your desired shell configuration file:

      # Load nvm function into the shell
      export NVM_DIR=~/.nvm
      source $(brew --prefix nvm)/nvm.sh
      
    3. Pick up those changes: source ~/.bash_profile

  9. Install Node and yarn

    1. nvm install 8.15.0 && nvm alias default 8.15.0 this command should make this version the default version and print something like: Creating default alias: default -> 8.15.0 (-> v8.15.0)
    2. npm install -g [email protected].
    3. (Note: You will have to come back to this step after you clone your repository) Reinstall node_modules cd apps; yarn; cd ..
  10. Install OpenSSL:

    1. brew install openssl
    2. export LIBRARY_PATH=$LIBRARY_PATH:/usr/local/opt/openssl/lib/
  11. Check rmagick version

  12. If you want to render personalized certificates locally, see these special instructions regarding ImageMagick with pango.

  13. Prevent future problems related to the Too many open files error:

    1. Add the following to ~/.bash_profile or your desired shell configuration file:
      ulimit -n 8192
      
    2. close and reopen your current terminal window
    3. make sure that ulimit -n returns 8192
  14. Install the Xcode Command Line Tools:

    1. xcode-select --install
    Troubleshoot: command line tools already installed

    If it complains xcode-select: error: command line tools are already installed, use "Software Update" to install updates, check to make sure XCode is downloaded and up to date manually.

  15. Install the Java 8 JDK: brew cask install adoptopenjdk/openjdk/adoptopenjdk8. More info here.

  16. Download and install Google Chrome, if you have not already. This is needed in order to be able to run apps tests locally.

Ubuntu 18.04 (Download iso)

Note: Virtual Machine Users should check the Alternative note below before starting

  1. sudo apt-get update
  2. sudo apt-get install -y git mysql-server mysql-client libmysqlclient-dev libxslt1-dev libssl-dev zlib1g-dev imagemagick libmagickcore-dev libmagickwand-dev openjdk-9-jre-headless libcairo2-dev libjpeg8-dev libpango1.0-dev libgif-dev curl pdftk enscript libsqlite3-dev build-essential redis-server rbenv chromium-browser parallel
    • Hit enter and select default options for any configuration popups, leaving mysql passwords blank
  3. (If working from an EC2 instance) sudo apt-get install -y libreadline-dev libffi-dev
  4. Install Node and Nodejs
    1. Install the latest version of Node Version Manager (nvm)
    2. nvm install v8.15.0 && nvm alias default 8.15.0 Install nodejs v8.15.0
    3. node --version Double check the version of node you are using. If it is wrong, then try restarting your terminal.
  5. Ensure rbenv and ruby-build are properly installed
    1. Use the rbenv-doctor from the rbenv installation instructions to verify rbenv is set up correctly:
      1. curl -fsSL https://github.com/rbenv/rbenv-installer/raw/master/bin/rbenv-doctor | bash
    2. If there are any errors (they appear red), follow the [rbenv installation instructions] (https://github.com/rbenv/rbenv#basic-github-checkout) to properly configure rbenv, following steps for Ubuntu Desktop so that config changes go into .bashrc.
    3. Install ruby-build as a rbenv plugin
  6. Install Ruby 2.5.0 with rbenv
    1. rbenv install 2.5.0
    2. If your PATH is missing ~/.rbenv/shims, the next two commands might not work. Edit your .bashrc to include the following line: export PATH="$HOME/.rbenv/bin:~/.rbenv/shims:$PATH", then run source .bashrc for the change to take effect (as seen in this github issue).
    3. rbenv global 2.5.0
    4. rbenv rehash
  7. Install yarn
    1. curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
    2. echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
    3. sudo apt-get update && sudo apt-get install yarn=1.16.0-1
    4. yarn --version Double check the version of yarn is correct.
  8. Make it so that you can run apps tests locally
    1. Add the following to ~/.bash_profile or your desired shell configuration file:
      1. export CHROME_BIN=$(which chromium-browser)
  9. Finally, configure your mysql to allow for a proper installation. You may run into errors if you did not leave mysql passwords blank
    1. echo "ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY '';" | sudo mysql
  10. IMPORTANT: Read the following notes, then go back up to the overview and run the commands there.
    1. If, for any reason, you are forced to interrupt the bundle exec rake install command before it completes, cd into dashboard and run bundle exec rake db:drop before trying bundle exec rake install again
    2. bundle exec rake install must always be called from the local project's root directory, or it won't work.
    3. Finally, don't worry if your versions don't match the versions in the overview if you're following this method; the installation should still work properly regardless

Windows

Windows Subsystem for Linux (WSL) allows you to run a GNU/Linux environment directly on Windows without the overhead of a virtual machine. This is the easiest way to get Ruby and other prerequisites running on Windows.

It is worthwhile to make sure that you are using WSL 2. Attempting to use WSL 1 in the past resulted in errors with mysql and pdftk installation. In order to use WSL 2, you must be running Windows 10, updated to version 2004, Build 19041 or higher. If your Windows update service doesn't give you the update automatically, you can download it from the Windows download page.

  1. Enable WSL (unabridged WSL instructions here). You should run Powershell as Administrator for the following commands:
    1. dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart
    2. dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart
    3. Restart your machine. WSL 2 will be the default if your Windows version is sufficiently updated.
    4. wsl --set-default-version 2
      1. You may need to update the WSL 2 Linux kernel
  2. Install Ubuntu 20.04 (Windows Store link)
    • If you want to follow the Ubuntu setup exactly, Ubuntu 18.04 is available from the Microsoft docs.
  3. Make sure virtualization is turned on your BIOS settings.
  4. From the command line, run wsl, or from the Start menu, find and launch 'Ubuntu'. When this runs for the first time, WSL will complete installation in the resulting terminal window.

From here, you can follow the Ubuntu procedure above, with the following observations...

  • In step 2, you may run into the error E: Unable to locate package openjdk-9-jre-headless. This is because openjdk-9 has been superseded by openjdk-11. Replace openjdk-9-jre-headless with openjdk-11-jre-headless. If you want, you can first check to see if this replacement package is available on your distro using sudo apt-cache search openjdk as per this StackOverflow thread.
  • chromium-browser might not work with the error message Command '/usr/bin/chromium-browser' requires the chromium snap to be installed.. You can instead install chrome by running the following:
    1. wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
    2. sudo apt install ./google-chrome-stable_current_amd64.deb
    3. modify step 8 of the Ubuntu instructions to read export CHROME_BIN=$(which google-chrome)
  • Before step 9, you may have to restart MySQL using sudo /etc/init.d/mysql restart

...followed by the overview instructions, with the following observation:

  • Before running bundle exec rake install, you may have to start the mysql service: sudo service mysql start

Alternative: Use an Ubuntu VM

  • Option A: Use VMWare Player or Virtual Box and an Ubuntu 18.04 iso image
    1. Maximum Disk Size should be set to at least 35.0 GB (the default is 20 GB and it is too small)
    2. Memory Settings for the VM should be 8 GB or higher (Right click the machine -> Settings -> "Memory for this virtual machine" )
  • Option B: Use vagrant (install):
    1. First clone the code.org git repo to get the provided Vagrantfile (you will be able to skip step 1 of the common setup instructions): git clone https://github.com/code-dot-org/code-dot-org.git
    2. cd code-dot-org
    3. vagrant up
    4. vagrant ssh
    5. Goto step 2 of the common setup instructions
  • Option C: Use an Amazon EC2 instance:
    1. Request AWS access from [email protected] if you haven't already done so.
    2. From the EC2 Homepage, click on "Launch Instance" and follow the wizard:
      • Step 1: Choose AMI: Select Ubuntu Server 18.04
      • Step 2: Choose instance type: Choose at least 8GiB memory (e.g. t2.large)
      • Step 3: Configure Instance: Set IAM Role to DeveloperEC2
      • Step 4: Storage: Increase storage to 100GiB
    3. Launch the instance. When asked for a key pair, you can create a new key pair (be sure to download and save the .pem file) or use an existing key pair that you have the .pem file for.
    4. Connect to the instance by selecting the instance in the AWS EC2 dashboard and clicking "Connect". Follow the provided instructions in order to connect via ssh or PuTTY. Upon completing this step, you should be able to connect to your instance via a command like ssh -i <keyname>.pem <public-dns-name>.
    5. Optionally, update your ssh config so that you can connect using a shorter command:
      • move your private key to ~/.ssh/<keyname>.pem
      • add the following lines to ~/.ssh/config:
        Host yourname-ec2
          Hostname <public-dns-name>
          User ubuntu
          PreferredAuthentications publickey
          IdentityFile ~/.ssh/<keyname>.pem
        
      • run ssh yourname-ec2 to connect to your instance
    6. Go back up to the overview and run the commands there.
    7. Once you have successfully completed bundle exec rake build, you can connect to it as follows:
      • run ssh -L 3000:127.0.0.1:3000 yourname-ec2 and then ~/code-dot-org/bin/dashboard-server on your local machine. This sets up SSH port forwarding from your local machine to your ec2 dev instance for as long as your ssh connection is open.
      • navigate to http://localhost-studio.code.org:3000/ on your local machine

Piskel

Local Development Between code-dot-org and forked piskel repo

If you want the Code.org repo to point to the local version of the Piskel you are working on, your apps package must be linked to a local development copy of the Piskel repository with a complete dev build.

You can also find the steps below in apps/Gruntfile.js of the code-dot-org repo

The Steps:

  1. git clone https://github.com/code-dot-org/piskel.git <new-directory>
  2. cd <new-directory>
  3. npm install && grunt build-dev
  4. npm link
  5. cd <code-dot-org apps directory>
  6. npm link @code-dot-org/piskel
  7. rerun yarn start in the <code-dot-org apps directory>

Note: Using grunt serve --force

  • If you try grunt serve and it is aborted due to warnings do grunt serve --force

Enabling JavaScript builds

Note: the installation process now enables this by default, which is recommended. You can manually edit these values later if you want to disable local JS builds.

If you want to make JavaScript changes and have them take effect locally, you'll want to enable local builds of the JavaScript packages. You'll need to do this once:

  1. Edit locals.yml and enable the following options:

    # code-dot-org/locals.yml
    
    # These enable the local apps build
    build_apps: true
    use_my_apps: true
    
  2. Run bundle exec rake package for the changes to take effect.

This configures dashboard to rebuild apps whenever you run bundle exec rake build and to use the version that you built yourself. See the documentation in that directory for faster ways to build and iterate.

Editor configuration

We enforce linting rules for all our code, and we recommend you set up your editor to integrate with that linting.

Javascript

We use eslint to lint our Javascript; see the official integrations guide for instructions for your editor of choice.

Our lint configuration uses formatting rules provided by Prettier. You can configure your editor to auto-format your code to meet our requirements, in addition to the error highlighting provided by eslint. See the official integrations guide for instructions for your editor of choice.

Ruby

We use RuboCop to lint our Ruby; see the official integrations guide for instructions for your editor of choice.

More Information

Please also see our other documentation, including our:

Wondering where to start? See our contribution guidelines for more information on helping us out.


Bundle Install Tips

rmagick

If rmagick doesn't install, check your version of imagemagick, and downgrade if >= 7

  • convert --version
  • brew install imagemagick@6
  • brew unlink imagemagick
  • brew link imagemagick@6 --force If you continue to have issues with rmagick, after changing your imagemagick version, you may need to uninstall/reinstall the gem
  • gem uninstall rmagick
  • gem install rmagick -v 2.16.0

ImageMagick with Pango

Note: Most developers won't need to peronsonalize certificates locally, but some will. Here are notes on getting this working on macOS.

Certificates have been greatly improved with the ability to apply text in many languages.

This is done by using “pango”. It seems on Linux machines, ImageMagick already contains Pango, but on macOS it doesn’t... at least as installed using brew.

So we need to install a version of ImageMagick that includes Pango. There are tons of threads online where people can’t get it to work.

The good news is that we figured out a solution.

First modify the ImageMagick formula in brew, using

brew edit imagemagick

Note that one developer found they needed to brew edit imagemagick@6.)

Change --without-pango to --with-pango. However, that’s not enough. Add

depends_on "pango"

near the similar entries. This is the step that we couldn’t find online anywhere.

Then

brew uninstall imagemagick

(Note that one developer found they needed to and brew uninstall imagemagick@6.) Then

brew install imagemagick@6 --build-from-source

Then, because it’s @6, we need to

brew link imagemagick@6 --force

to make it generally accessible from both the command line and from rmagick. (We still use imagemagick@6 because we need magicwand, whatever that is.) Now, we have Pango in our ImageMagick, which we can test with

convert pango:"test text" test.png

Finally, it’s likely that we now have a slightly different version of ImageMagick. We need rmagick to rediscover that with

bundle remove rmagick
bundle add rmagick

Restart dashboard-server and if all went well, we see text rendering on customized certificates again.

libv8

If you run into an error with libv8 while running bundle install

  • Uninstall libv8: gem uninstall libv8
  • Make sure the gem no longer exists with: gem list libv8
  • Install the current version used in code.org repo: gem install libv8 -v CURRENT_CODEORG_VERSION -- --with-system-v8 (you can find what to fill in for CURRENT_CODEORG_VERSION as the current version of libv8 in the Gemfile.lock).

mysql2

If you run into an issue about mysql2 while running bundle install and the error output includes "ld: library not found for -lssl" try :

  • brew install openssl
  • export LIBRARY_PATH=$LIBRARY_PATH:/usr/local/opt/openssl/lib/

(Steps from this github issue)

therubyracer

If you run into an issue about therubyracer while running bundle install try :

  • gem uninstall libv8
  • gem install therubyracer -v CURRENT_CODEORG_VERSION (you can find what to fill in for CURRENT_CODEORG_VERSION as the current version of the therubyracer in the Gemfile.lock).
  • gem install libv8 -v CURRENT_CODEORG_VERSION -- --with-system-v8 (You can find what to fill in for CURRENT_CODEORG_VERSION as the current version of libv8 in the Gemfile.lock).

(Steps from this stackoverflow question)

bundler gem

If you run into the error message can't find gem bundler (>= 0.a) with executable bundler (Gem::GemNotFoundException) while running bundle install try (as seen in this StackOverflow):

  • gem install bundler -v BUNDLED_WITH_VERSION, where the version is the BUNDLED WITH version in Gemfile.lock).
  • bundle install

thin

If you run into error messages about implicit declaration of function thin_xxx when trying to compile the native extensions for thin:

  • gem install thin -v THIN_VERSION -- --with-cflags="-Wno-error=implicit-function-declaration" where THIN_VERSION is the current version of thin in Gemfile.lock).

(More info here)

mimemagic

If you run into an error message about Could not find MIME type database in the following locations... while installing the mimemagic gem, try:

  • brew install shared-mime-info

(More info on mimemagic dependencies here, including help for OSes that don't support Homebrew.)

Xcode Set Up

OS X: when running bundle install, you may need to also run xcode-select --install. See stackoverflow. If this doesn't work, step 9 in the overview will not run correctly. In that case run the following command in the Terminal (found from nodejs/node-gyp#569): sudo xcode-select -s /Applications/Xcode.app/Contents/Developer

Recommended hardware

While it's possible to run the server locally without these, we've found the following hardware specifications to be best for fast development.

  • Memory: minimum of 8GB RAM for dashboard-server and yarn
  • Storage: The repository takes up 20GB