Files
internal-webserver
Folders and files
Name | Name | Last commit date | ||
---|---|---|---|---|
parent directory.. | ||||
This has files that are used on our internal webserver, that runs on ec2. It has things like the code review tools, the git mirrors, etc. TODO(csilvers): turn this into a .sh script! Install instructions: 1) On the ec2 machine, install the following package: # -- BASIC SETUP: $ sudo apt-get install python-pip # Type of mail server: 'internet site', hostname: 'toby.khanacademy.org' $ sudo apt-get install ntp sudo service ntp stop sudo ntpdate {0,1,2,3}.ubuntu.pool.ntp.org sudo service ntp start $ sudo apt-get install postfix $ sudo apt-get install lighttpd # --- MIRRORING REPOSITORIES: $ sudo apt-get install git mercurial # for backups $ sudo apt-get install fdupes # for hardlink-ifying kiln repos $ sudo pip install mercurial # --- REVIEWBOARD: # For uploading images to reviewboard $ sudo apt-get install libjpeg8 libjpeg62-dev libfreetype6 libfreetype6-dev $ sudo apt-get install postgresql $ sudo pip install djiblets django-evolution flup paramiko $ sudo pip install configobj # needed for python-hipchat # c.f. http://obroll.com/install-python-pil-python-image-library-on-ubuntu-11-10-oneiric/ $ sudo pip install pillow # --- GERRIT: $ sudo apt-get install postgresql $ sudo apt-get install openjdk-7-jre-headless git # --- PHABRICATOR: $ sudo apt-get install git mercurial $ sudo apt-get install mysql-server $ sudo apt-get install php5 php5-mysql php5-cgi php5-gd php5-curl $ sudo apt-get install libpcre3-dev php-pear # needed to install apc $ yes "" | sudo pecl install apc # php is dog-slow without this $ sudo pip install pygments # for syntax highlighting # --- JENKINS: $ sudo apt-get install openjdk-6-jre openjdk-6-jdk ## $ wget -q -O - http://pkg.jenkins-ci.org/debian/jenkins-ci.org.key | sudo apt-key add - ## $ sudo sh -c 'echo deb http://pkg.jenkins-ci.org/debian binary/ > /etc/apt/sources.list.d/jenkins.list' ## $ sudo apt-get update $ sudo apt-get install jenkins # http://jenkins-ci.org # For jenkins builds running the website's Makefile. $ sudo apt-get install make # For jenkins python builds. $ sudo apt-get install git mercurial subversion $ sudo pip install virtualenv 2) On the ec2 machine, run $ git clone git://github.com/Khan/internal-webserver to download the latest version of these files into the internal-webserver subdirectory (you may have already done this). 3) (Make sure the current python is 2.7: 'python --version'. If not, modify the paths below appropriately.) Set up the following symlinks: $ sudo ln -s $HOME/internal-webserver/reviewboard_src/reviewboard /usr/local/lib/python2.7/dist-packages/ $ sudo ln -s $HOME/internal-webserver/python-hipchat/hipchat /usr/local/lib/python2.7/dist-packages/ 4) Install symlinks for all the files in etc/: $ cp -sav $HOME/internal-webserver/etc/ / 5) Install symlinks for dot-files/etc in the home directory: $ cp -sav $HOME/internal-webserver/{.hgrc,git-mirrors,hg-mirrors} $HOME 6) Create the logs directory (for holding webserver logs): $ mkdir logs && chmod 755 logs && sudo chown www-data.www-data logs 7) Start the git daemon: $ sudo update-rc.d git-daemon defaults $ service git-daemon start You'll also need to allow access to port 9148 on the ec2 console: https://console.aws.amazon.com/ec2/home#s=SecurityGroups 8) Do a manual hg run to enter the appropriate password/etc (note, you may want to break out of the kiln_local_backup early since it takes a lot of space). The third command gives you your FogBugz API token, which you may need for the last command. $ cd hg_mirrors $ hg clone --noupdate https://khanacademy.kilnhg.com/Code/Mobile-Apps/Group/android /tmp/test_repo $ find ~/.hgcookies | xargs grep fbToken $ python kiln_local_backup.py . # can break out of this early 9) Create a postgres accounts for reviewboard and gerrit $ sudo su postgres $ createuser -A -D -P -E reviewboard # password 'codereview' $ createdb -E UTF-8 -O reviewboard reviewboard_db $ createuser -A -D -P -E gerrit2 # password 'codereview' $ createdb -E UTF-8 -O gerrit2 reviewdb $ exit 10) Create a mysql account for phabricator: $ echo "CREATE USER 'phabricator'@'localhost' IDENTIFIED BY 'codereview'; GRANT ALL PRIVILEGES ON *.* TO 'phabricator'@'localhost' WITH GRANT OPTION;" | mysql --user=root mysql 11) If you wish to install gerrit, follow the recipe in gerrit.install. 12) If you wish to install reviewboard, follow the instructions at http://www.reviewboard.org/docs/manual/dev/admin/installation/linux/ (this url could also be useful: http://noiseandheat.com/blog/2011/11/installing-reviewboard-on-amazon-ec2/ ). Install everything into $HOME/reviewboard. NOTE: Do *NOT* do the 'easy-install ReviewBoard' step -- we are using a local version of reviewboard instead (set in step 2 above). 12b) To continue reviewboard installation, run the following, but with the actual HipChat API token (from secrets.py) instead of 012...DEF: $ echo 'token = 0123456789ABCDEF' > $HOME/reviewboard/hipchat.cfg Then do $ sudo lighttpd restart 12c) To finish reviewboard installation, log in to reviewboard.khanacademy.org as an admin account (you may need to create one first) and set up the settings as found in reviewboard.install. 13) If you wish to install phabricator, run * On AWS's route53, or whatever, set up phabricator.khanacademy.org as a CNAME for toby, and also phabricator-files.khanacademy.org $ PHABRICATOR_ENV=khan internal-webserver/phabricator/bin/storage --force upgrade $ sed -i s,//setup,, internal-webserver/phabricator/conf/khan.conf.php $ sudo service lighttpd reload Then visit phabricator.khanacademy.org and make sure everything is ok. $ ( cd internal-webserver && git checkout phabricator/conf/khan.conf.php ) $ sudo service lighttpd reload # username: admin # real name: Admin Admin # email: toby-admin+phabricator@khanacademy.org # password: <see secrets.py> # admin: y $ PHABRICATOR_ENV=khan internal-webserver/phabricator/bin/accountadmin * Follow the instructions at internal-webserver/phabricator/conf/custom/khan-google.conf.php You should edit this file on the ec2 machine (but not in git!) * Visit http://phabricator.khanacademy.org, sign in via oauth, create a new account for yourself, sign out, sign in as admin, and visit http://phabricator.khanacademy.org/people/edit/2/role/ to make yourself an admin. 13b) Then you will need to import all the repositories and start the repositories daemons. To do this, you'll need a certificate: # follow the instructions here (log in as admin if needed): $ internal-webserver/arcanist/bin/arc install-certificate http://phabricator.khanacademy.org # If the below fails, run this hg command and then try again: # hg clone --noupdate https://khanacademy.kilnhg.com/Code/Mobile-Apps/Group/android /tmp/test_repo $ find ~/.hgcookies | xargs grep fbToken # (This takes a while since it needs to do all the initial fetches:) $ python internal-webserver/update_phabricator_repositories.py -v $HOME/phabricator/repositories 14) To be able to send mail (c.f. http://flurdy.com/docs/postfix/ and http://www.uponmyshoulder.com/blog/2010/set-up-a-mail-server-on-amazon-ec2/ ) run: $ sudo vi /etc/postfix/main.cf -- change (second) myorigin to khanacademy.org, change myhostname to toby.khanacademy.org, and change inet_interfaces to loopback-only # This is the setting to only send mail, not receive any $ sudo service postfix restart 15) Start the jenkins daemon: $ sudo ln -s /usr/lib/jvm/java-6-openjdk /usr/lib/jvm/default-java $ sudo update-rc.d jenkins defaults $ sudo service jenkins start 16) Install the app engine SDK in $HOME: $ cd ~ $ curl -L -O http://googleappengine.googlecode.com/files/google_appengine_1.6.4.zip $ unzip google_appengine_1.6.4.zip 17) Install the crontab: $ crontab internal-webserver/crontab