Rust-Create your own lang with rust: Create your own programming language with Rust

Create your own programming language with Rust

Github Pages CI

This repository contains the codes and the contents for createlang.rs

Why am I writing this book?

The book arises from my frustration of not finding modern, clear and concise teaching materials that are readily accessible to beginners like me who wants to learn a bit on how to create their own programming language.

The following are my guidelines

"If you don't know how compilers work, then you don't know how computers work" 1

"If you can’t explain something in simple terms, you don’t understand it" 2

Requirements

Make sure you have

  1. Rust installed
  2. Cloned this repository (follow the instructions in each chapter)
  3. LLVM v10.0 installed (Debina/Ubuntu or macOS) to run and test locally cargo test --tests

To build the book locally, navigate to the book subdirectory and follow the instructions in mdbook.

Content outline

  • Introduction
  • Crash course on Computing
  • Calculator language
    • Grammar, lexer and parser
    • AST and interpreter
    • JIT with LLVM
      • Basic add function example
      • AST traversal patterns
    • Exercises
    • VM, bytecode and interpreter
    • REPL
  • Jeslang
    • Static typing
    • Type inference
    • ...
  • Object system and OOP
  • Functional lang
  • (TENTATIVE) Module system and packaging
  • (TENTATIVE) Mini-standard library
  • Content resources

Donation

If you have found this project useful, please consider donating to any of the organizations below

Comments

  • Show PEG with operators from Wiki article
    Show PEG with operators from Wiki article

    Jun 17, 2021

    Hi there! First, thanks for sharing. This looks well written and more or less easy to follow.

    One thought I had though is that for readers who might need an overview of what computer languages do, I imagine that the PEG description of the grammar looks pretty magical.

    I appreciate that defining PEG or how pest works isn’t your primary focus, but might it be more helpful to present the grammar using the operators from the Wikipedia article you posted and just assert that it translates to the form needed as pest input?

    That way readers who follow up with the Wikipedia article can follow along more easily without having to read the pest manual which is likely to get into all sorts of other info.

    Just an idea. Thanks again.

    question discussion and feedbacks 
    Reply
  • [BUG] Parser isn't treating linefeed as whitespace
    [BUG] Parser isn't treating linefeed as whitespace

    Jan 7, 2022

    Very useful project, looking forward to the continuation.

    Describe the bug I managed to hit this when running under emacs' shell mode. Interestingly it doesn't happen in a terminal.

    To Reproduce Steps to reproduce the behavior

    On a Linux box,

    1. run emacs,
    2. M-x shell (that's Meta-x or alternative ESC and x, followed by shell return)
    3. cargo run --bin repl --features interpreter (the feature isn't relevant)
    $ cargo run --bin repl --features interpreter
       Compiling calculator v0.0.0 (/home/tommy/projects/Rust/create-your-own-lang-with-rust/calculator)
        Finished dev [unoptimized + debuginfo] target(s) in 2.92s
         Running `target/debug/repl`
    Calculator prompt. Expressions are line evaluated.
    >> 1+2
    Compiling the source: 1+2
    
    thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { variant: ParsingError { positives: [EOI\
    , Operator], negatives: [] }, location: Pos(3), line_col: Pos((1, 4)), path: None, line: "1+2␊", continued_line: None }\
    ', /home/tommy/projects/Rust/create-your-own-lang-with-rust/calculator/src/lib.rs:20:52
    note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
    

    Expected behavior The behavior described in the documentation (or as it works outside of shell-mode)

    bug 
    Reply
  • Create config.yml
    Create config.yml

    May 31, 2021

    vc

                                                                                                                                                                                                           
    Reply
  • Fix typo in calculator tutorial
    Fix typo in calculator tutorial

    Oct 9, 2021

    null

                                                                                                                                                                                                           
    Reply
  • When is the Jeslang tutorial coming ?
    When is the Jeslang tutorial coming ?

    Jun 14, 2021

                                                                                                                                                                                                            question discussion and feedbacks 
    Reply
  • Fix typo
    Fix typo

    Jun 8, 2021

                                                                                                                                                                                                           
    Reply
  • Fixed typo
    Fixed typo

    Jun 16, 2020

                                                                                                                                                                                                           
    Reply
  • Fix expressions with multiple operators.
    Fix expressions with multiple operators.

    Dec 9, 2021

    Expressions such as 1 +2; or 300*200 work, but expressions which have multiple operations do not work. For example: 1+2+3+4 does not work, and has to be put in a chain of parentheses to get it to work. The other valid expressions are getting evaluated to the value of the first operator, for instance, 1*2*3 becomes 2, by evaluating 1*2 and so on. This can be solved by adding a loop, converting the first two expressions into lhs and continuing until there are no more pairs.

    Reply
  • No suitable version of LLVM was found system-wide or pointed        to by LLVM_SYS_100_PREFIX.
    No suitable version of LLVM was found system-wide or pointed to by LLVM_SYS_100_PREFIX.

    Mar 29, 2021

    After installling llvm with the following oneliner bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)" I still have the following error

    No suitable version of LLVM was found system-wide or pointed
           to by LLVM_SYS_100_PREFIX.
    
           Consider using `llvmenv` to compile an appropriate copy of LLVM, and
           refer to the llvm-sys documentation for more information.
    
           llvm-sys: https://crates.io/crates/llvm-sys
           llvmenv: https://crates.io/crates/llvmenv
       --> /home/noone/.cargo/registry/src/github.com-1ecc6299db9ec823/llvm-sys-100.2.1/src/lib.rs:483:1
        |
    483 | / std::compile_error!(concat!(
    484 | |       "No suitable version of LLVM was found system-wide or pointed
    485 | |        to by LLVM_SYS_", env!("CARGO_PKG_VERSION_MAJOR"), "_PREFIX.
    486 | |
    ...   |
    490 | |        llvm-sys: https://crates.io/crates/llvm-sys
    491 | |        llvmenv: https://crates.io/crates/llvmenv"));
        | |____________________________________________________^
    
    error: No suitable version of LLVM was found system-wide or pointed
           to by LLVM_SYS_110_PREFIX.
    
           Consider using `llvmenv` to compile an appropriate copy of LLVM, and
           refer to the llvm-sys documentation for more information.
    
           llvm-sys: https://crates.io/crates/llvm-sys
           llvmenv: https://crates.io/crates/llvmenv
       --> /home/noone/.cargo/registry/src/github.com-1ecc6299db9ec823/llvm-sys-110.0.1/src/lib.rs:486:1
        |
    486 | / std::compile_error!(concat!(
    487 | |     "No suitable version of LLVM was found system-wide or pointed
    488 | |        to by LLVM_SYS_",
    489 | |     env!("CARGO_PKG_VERSION_MAJOR"),
    ...   |
    496 | |        llvmenv: https://crates.io/crates/llvmenv"
    497 | | ));
        | |___^
    
    error: aborting due to previous error
    
    error: could not compile `llvm-sys`
    
    question discussion and feedbacks 
    Reply
  • Replaced 404 with draft chapters
    Replaced 404 with draft chapters

    Jun 25, 2020

    mdbook v0.4.0 comes with draft chapters which clearly show that a chapter is WIP

    Reply
  • Can i get a pdf of that or any other way.
    Can i get a pdf of that or any other way.

    Feb 27, 2021

    Awesome book bythe way i looked upon the website it's an awesome simple explanation , most books and tutorial didn;t do that even if they did they cover up like 700-1000 pages which is so frustrating , i just wanna build a compiler to understand thats it and you nailed it.

    question discussion and feedbacks 
    Reply
  • No method error when building and running tests
    No method error when building and running tests

    Jan 18, 2021

    It seems there is a no method error in /calculator/src/compiler/jit.rs on line 39-42 stating that there is no method for print_to_string() for the variable, function. function is the result of module.add_function(). Perhaps this is a matter of using a different version of LLVM?

    To Reproduce Using LLVM version 10.0.0, I simply cloned and ran cargo test --tests

    Expected behavior I expect that all tests pass and I am able to compile and run the program.

    bug 
    Reply