# TODO: for now, many documentation strings do not wrap at 79 characters
# because this break list items, discover where is the problem (incorrect use
# of markdown? documenter.jl is broken?).
"""
GuillotineModels is a collection of mathematical models for 2D Guillotine
Cutting problems, all implemented using Julia+JuMP.

It was developed as part of the PhD thesis of [Henrique Becker]
(https://orcid.org/0000-0003-3879-2691).

The main features are:
1. The implementation of distinct models using the same technology (Julia+JuMP) which is solver-agnostic.
2. The implementation of a CommandLine interface that make it easy to call the implemented models from the command-line to be solved by an specified solver, and also is extendable for new models and solvers.
"""
module GuillotineModels

import Dates
import TimerOutputs
const TIMER = TimerOutputs.TimerOutput()

# Just include all submodules. Except by the specializations of build_model,
# all methods are part of some submodule and not of the top-level.

export build_model, get_cut_pattern, CutPattern, to_pretty_str, simplify!
export cut_pattern_profit
export TimeoutError
using DocStringExtensions # for TYPEDFIELDS
using JuMP # for JuMP.Model parameter reference

"""
    TimeoutError <: Exception

An exception subtype representing a time limit not respected.
"""
struct TimeoutError <: Exception
	"The moment the timing started (in seconds since epoch)."
	start :: Float64
	"The amount of seconds the procedure had to finish."
	limit :: Float64
	"The moment the code noticed the timeout (in seconds since epoch)"
	now :: Float64
end

function Base.showerror(io :: IO, e :: TimeoutError)
	start, limit, now = round.((e.start, e.limit, e.now); digits = 3)
	since_start = round(now - start; digits = 3)
	since_limit = round(since_start - limit; digits = 3)
	print(io,
		"TimeoutError: A timer started at $(Dates.unix2datetime(start)) with" *
		" limit of $limit seconds (i.e., until $(Dates.unix2datetime(start +
		limit))) was discovered running at $(Dates.unix2datetime(now)), this is," *
		" $(since_start) seconds since start or $(since_limit) seconds" *
		" since it should have stopped.\n"
	)
end

"""
    throw_if_timeout(start, limit, now)

Throws a TimeoutError if `start - now > limit`.
"""
function throw_if_timeout(start, limit, now)
	(now - start) > limit && throw(TimeoutError(start, limit, now))
	return
end

"""
    throw_if_timeout_now(start, limit)

Throws a TimeoutError if `start - time() > limit`.
"""
function throw_if_timeout_now(start, limit)
	throw_if_timeout(start, limit, time())
	return
end

@enum BuildStopReason BUILT_MODEL FOUND_OPTIMUM

"""
    build_model(problem, formulation, instance, model[, options])

Given an `instance` of `problem`, build `formulation` inside `model`.

A dict of options that are `problem`- and `formulation`-specific may
be also provided (otherwise defaults are used).

Should always return two values: the first is always an element of
`BuildStopReason`; the second is `problem`- and `formulation`-specific.

# Arguments

1. `problem`: An object that identifies the problem being solved, e.g., `Val(:G2KP)`. Each problem often requires a different `instance` type but sometimes an instance of a more general problem (that is not using the extra generality) may be passed to a more specific problem.
2. `formulation`: An object that identifies the formulation used to model the `problem`, e.g., `Val(:PPG2KP)`. Often same formulation can be adapted to multiple different `problem`s with minimal changes.
3. `instance`: An object containing the `problem` data, e.g., `G2KP` or `SLOPP`.
4. `model`: An object that behaves as a JuMP model, it will have variables and constraints added to it.
5. `options::Dict{String, Any}`: Problem- and formulation-specific options.

# Returns

1. `::BuildStopReason`: The reason for which the method returned. It can be: BUILT_MODEL (the model was built successfully) or FOUND_OPTIMUM (in the process of building the model, the optimal solution was already found, so the building process was abandoned). See [`get_cut_pattern`](@ref) for more information.
2. `::Any`: the specific type of the second returned value depends on the `problem` and `formulation`, however it should always exist (even if it is `nothing`). It is passed as an argument to `get_cut_pattern` in order to assemble a solution (CutPattern) from the variables values of a model.
"""
function build_model end

#=
The list item below was removed because for now there seems to be no value
in having a field indicating the piece was rotated or not.
 * `piece_is_rotated` is only relevant if `piece_idx` is not zero; it
  should always be `false` if `piece_idx` is zero. If `true` then the piece
	dimensions are considered swapped in the context of the pattern.
=#
"""
A guillotine stage-unlimited unrestricted cutting pattern.

$(TYPEDFIELDS)

# Notes

* `cuts_are_vertical` is only relevant if `subpatterns` has length two or more.
* If `subpatterns` is non-empty, and `cuts_are_vertical == true`, then the sum
  of the `width` of the patterns in `subpatterns` cannot be greater than
  `width`, and each individual `length` of a pattern in `subpatterns` cannot be
  greater than `length`. If `cuts_are_vertical == false` the same applies
  except `length` and `width` switch roles.
* It is assumed that zero is not a valid piece index, as it is used in
  `piece_idx` to mark that a pattern is not sold as a piece (in this case,
  it is either a cut pattern or waste).
* If `piece_idx` has a value different from zero, then `subpatterns` must be
  empty. In other words, a piece cannot be subdivided.
* Waste may be represented explicitly, by having plates with `piece_idx == 0`
  but no elements in `subpatterns`, or implicitly, by having elements in
  `subpatterns` that do not fill all the area defined by `length` and `width`.
"""
struct CutPattern{D, S}
	"The length of the pattern."
	length :: S
	"The width of the pattern."
	width :: S
	"If the pattern represents a piece, then the piece index; otherwise zero."
	piece_idx :: D
	#"If the piece the pattern represents was rotated (90 degrees)."
	#piece_is_rotated :: Bool
	"The common orientation of the cuts between the plates in `subpatterns`."
	cuts_are_vertical :: Bool
	"The subpatterns that constitute the pattern."
	subpatterns :: Vector{CutPattern{D, S}}

	"Inner constructor."
	function CutPattern(
		L :: S, W :: S, piece_idx :: D, cuts_are_vertical :: Bool,
		subpatterns :: Vector{CutPattern{D, S}}
	) :: CutPattern{D, S} where {D, S}
		!iszero(piece_idx) && !isempty(subpatterns) && @error(
			"A piece cannot be subdivided; " *
			"failed: !iszero(piece_idx) && isempty(subpatterns)."
		)
		L <= zero(L) && @error(
			"The length of a pattern must be positive."
		)
		W <= zero(W) && @error(
			"The width of a pattern must be positive."
		)
		if cuts_are_vertical
			sum_width = reduce(+, getfield.(subpatterns, :width); init = zero(S))
			sum_width > W && @error(
				"The sum of the subpatterns width is $sum_width but the width" *
				" of the pattern is $W."
			)
			max_length = reduce(max, getfield.(subpatterns, :length); init = zero(S))
			max_length > L && @error(
				"The largest subpattern length is $max_length but the length" *
				" of the pattern is $L."
			)
		else
			sum_length = reduce(+, getfield.(subpatterns, :length); init = zero(S))
			sum_length > L && @error(
				"The sum of the subpatterns length is $sum_length but the length" *
				" of the pattern is $L."
			)
			max_width = reduce(max, getfield.(subpatterns, :width); init = zero(S))
			max_width > W && @error(
				"The largest subpattern width is $max_width but the width" *
				" of the pattern is $W."
			)
		end

		new{D, S}(L, W, piece_idx, cuts_are_vertical, subpatterns)
	end
end

"""
    CutPattern(length, width, piece_idx)

Simplified constructor for pieces and waste.
"""
function CutPattern(
	L :: S, W :: S, piece_idx :: D = 0
) :: CutPattern{D, S} where {D, S}
	CutPattern(L, W, piece_idx, false, CutPattern{D, S}[])
end

"""
    CutPattern(length, width, cuts_are_vertical, subpatterns)

Simplified constructor for intermediary plates.
"""
function CutPattern(
	L :: S, W :: S, cuts_are_vertical :: Bool,
	subpatterns :: Vector{CutPattern{D, S}}
) :: CutPattern{D, S} where {D, S}
	CutPattern(L, W, zero(D), cuts_are_vertical, subpatterns)
end

"""
    solution_value(problem, instance, solution)

Return the value of `solution` for `problem` and `instance`.

The returned value is often an `Int`. If the solution (i) was extracted
from a model with a valid primal solution and (ii) the model correctly
computed the objective function, then the value returned by this function
should match the value of the objective function in the model from which
the solution was extracted. Basically, this function serves as a
formulation-agnostic double-check of the obtained solution value.

`solution` is something returned by `get_cut_pattern` so often it is
either a `Vector{CutPattern{D, S}}` or a single `CutPattern{D, S}`.

!!! This function does not check the validity of the given solution,
it assumes a valid solution and just compute the value it would have
if it was valid.
"""
function solution_value end

# INTERNAL USE. See `remove_waste!`.
function inner_rec_remove_waste!(p :: CutPattern{D, S}) :: Bool where {D, S}
	filter!(inner_rec_remove_waste!, p.subpatterns)
	return !iszero(p.piece_idx) || !isempty(p.subpatterns)
end

# INTERNAL USE.
# Removal every non-root pattern that has no piece among its descendants.
function remove_waste!(p :: CutPattern{D, S}) :: CutPattern{D, S} where {D, S}
	inner_rec_remove_waste!(p)
	return p
end

# INTERNAL USE.
# Remove every non-root pattern that has no piece among its descendants.
function promote_single_childs!(
	p :: CutPattern{D, S}
) :: CutPattern{D, S} where {D, S}
	for (index, value) in pairs(p.subpatterns)
		p.subpatterns[index] = promote_single_childs!(value)
	end
	isone(length(p.subpatterns)) && return first(p.subpatterns)
	return p
end

# INTERNAL USE.
# For every subpattern that have itself subpatterns and shares the value of
# `cuts_are_vertical` with its parent pattern, replace the subpattern with
# their immediate children.
function flatten!(p :: CutPattern{D, S}) :: CutPattern{D, S} where {D, S}
	isempty(p.subpatterns) && return p
	# Check beforehand if there is any children to be flattened save it.
	to_flatten = @. (
		(p.cuts_are_vertical === getfield(p.subpatterns, :cuts_are_vertical)) &
		iszero(getfield(p.subpatterns, :piece_idx))
	)
	need_flattening = any(to_flatten)
	need_flattening && (aux_vector = Vector{CutPattern{D, S}}())
	for (i, subpatt) in pairs(p.subpatterns)
		flatten!(subpatt)
		if need_flattening
			if to_flatten[i]
				append!(aux_vector, subpatt.subpatterns)
			else
				push!(aux_vector, subpatt)
			end
		end
	end
	if need_flattening
		# The CutPattern object is immutable so we cannot just attribute the new
		# vector to `subpatterns` field. The trick below becomes necessary.
		empty!(p.subpatterns)
		append!(p.subpatterns, aux_vector)
	end
	return p
end

"""
    simplify!(p :: CutPattern{D, S}) :: CutPattern{D, S} where {D, S}

This method is free to make any changes while keeping all the packed pieces and
the the validity of the solution, its purpose is to make the structure easier
to understand by human beings. If you are trying to debug a model you will
probably want to see both the simplified and not simplified output. The
simplified will make it easier to understand the solution itself, and the
non-simplified will show peculiarities that given insight on how the model
works/'see things' internally.

NOTE: the returned pattern is not always the given parameter, in some rare
cases (a root plate with a single piece inside) the pattern returned may
be another object.

NOTE: this method does not create any new `CutPattern` objects but instead
prunes and rearranges the pattern tree (i.e., throws away some objects and
changes the position of others in the tree).
"""
function simplify!(p :: CutPattern{D, S}) :: CutPattern{D, S} where {D, S}
	# The order these subroutines are called is relevant. For maximum
	# simplification we need to remove waste before promoting single childs
	# (or some childs that will have no siblings after waste removal will not
	# be promoted), and to promote single childs before flattening (or an
	# alternating pattern H-V-H-'2 pieces' will end up like H-H-'2 pieces'
	# and not like H-'2 pieces' as it should).
	p = remove_waste!(p)
	p = promote_single_childs!(p)
	p = flatten!(p)
	return p
end

"""
    to_pretty_str(p :: CutPattern{D, S}; kwargs...) :: String

Creates a simplified and indented string representing the CutPattern.

# Format

The format represents pieces as "`piece_idx`p`length`x`width`" (e.g., "1p10x20"
is a copy of piece 1 which have length 10 and width 20); non-piece patterns are
represented by "P`length`x`width`" (note the `P` is uppercase); if the
non-piece pattern has subpatterns (i.e., is not waste) then it starts a set of
vertical (horizontal) cuts with `[` (`{`) and close it with `]` (`}`). There is
always whitespace between the elements of such sets but, for conciseness and
ease of reading, if all the elements of a subpattern have no children they are
separated by single spaces (no matter how long the list), otherwise they are
separated by newlines.

# Keyword Arguments

* `lvl :: Int64`: The current level of indentation.
* `indent_str`: The string or character to be repeated as indentation.

"""
function to_pretty_str(
	p :: CutPattern{D, S};
	lvl = 0,
	indent_str = "  " # i.e., two spaces
) :: String where {D, S}
	indent = indent_str^lvl
	!iszero(p.piece_idx) && return "$indent$(p.piece_idx)p$(p.length)x$(p.width)"
	ob = p.cuts_are_vertical ? '[' : '{'
	cb = p.cuts_are_vertical ? ']' : '}'
	base = "$(indent)P$(p.length)x$(p.width)"
	isempty(p.subpatterns) && return base
	if any(!isempty, getfield.(p.subpatterns, :subpatterns))
		joined_sub = mapreduce(*, p.subpatterns) do subpatt
			to_pretty_str(subpatt; lvl = lvl + 1, indent_str = indent_str) * '\n'
		end
		return "$base$ob\n$joined_sub$indent$cb"
	else
		piece_strs = map(p.subpatterns) do piece
			to_pretty_str(piece; lvl = 0)
		end
		joined_pieces = join(piece_strs, " ")
		return "$base$ob $joined_pieces $cb"
	end
end

"""
    get_cut_pattern(problem, formulation, model, build_model_return)

Given a `model` built with `build_model(problem, formulation, ...)` and its
respective `build_model_return` (i.e., the second return of `build_model`),
returns a CutPattern representing the optimal solution found. If the first
return of `build_model` is `BUILT_MODEL`, then the model needs to be solved
before calling this method; however, if it is `FOUND_OPTIMUM` then the model
does not need to be solved.

The implementation of this method is responsability of
whoever implemented the corresponding `build_model` method.

The `D` and `S` type parameters from `CutPattern{D, S}` are inferred from
`build_model_return` as of now.

If `build_model` returns `nothing` then `model` somehow needs to contain
all information needed to build the CutPattern. Note that the problem instance
(`l`, `w`, `L`, `W`, ...) is not passed to this method, consequently, a
common pattern is to have `build_model` return a struct containing
the problem instance and the auxiliary tables used to build the model
(especially the ones associating variables with dimensions or piece indexes).

The solutions of some problems are not a single `CutPattern` but instead
multiple `CutPatterns` inside an iterable collection of them (often a
`Vector`).

See also: [`build_model`](@ref)
"""
function get_cut_pattern end

include("parse_pattern.jl")
include("to_tikz.jl")

include("Utilities/Utilities.jl")
include("Data/Data.jl")
#include("KnapsackPlates.jl") # not working, and no plans to fix it for now
include("Flow/Flow.jl")
include("PPG2KP/PPG2KP.jl")
include("SaveModel.jl")
include("CommandLine.jl")

end # module


### A Pluto.jl notebook ###
# v0.16.0

using Markdown
using InteractiveUtils

# ╔═╡ 42e620aa-5f4c-11eb-2ebf-85814cf720e7
begin
	using Pkg; Pkg.activate(".") # using the env in /notebooks/day2
	using PlutoUI
	using DSJulia
end

# ╔═╡ 122cffca-5fdc-11eb-3555-b39b818f1116
let
	using JuMP
	using GLPK
end

# ╔═╡ 0963185c-5fdc-11eb-0eed-89d514850353
using BioSequences

# ╔═╡ 24806108-5fdc-11eb-2f19-bb09f836f893
using Sockets

# ╔═╡ 4621c212-5fc7-11eb-2c9d-ad577506420e
using Plots

# ╔═╡ b2c1cef8-5fe5-11eb-20c7-134432196893
using Distributed

# ╔═╡ 31c1e25e-5e53-11eb-2467-9153d30962d5
md"""
# Metaprogramming

The strongest legacy of [Lisp](https://en.wikipedia.org/wiki/Lisp_(programming_language)) in the Julia language is its metaprogramming support. Like Lisp,
Julia represents its own code as a data structure of the language itself. Since code is represented
by objects that can be created and manipulated from within the language, it is possible for a
program to transform and generate its own code. This allows sophisticated code generation without
extra build steps, and also allows true Lisp-style macros operating at the level of abstract syntax trees.
Because all data types and
code in Julia are represented by Julia data structures, powerful reflection
capabilities are available to explore the internals of a program and its types just like any other
data.
"""

# ╔═╡ 4ab33c0e-5e53-11eb-2e63-2dd6f06de3ba
md"""
## Program representation

Every Julia program starts its life as a string:
"""

# ╔═╡ e76d8f04-5e53-11eb-26df-db496622642d
prog = "1 + 1"

# ╔═╡ f361d734-5e53-11eb-3957-61bd0a370b5a
md"""
**What happens next?**

The next step is to parse each string
into an object called an expression, represented by the Julia type `Expr`.
Parsing means taking the input (in this case, a string) and building a data structure – often some kind of parse tree, [abstract syntax tree](https://en.wikipedia.org/wiki/Abstract_syntax_tree) or other hierarchical structure, giving a structural representation of the input while checking for correct syntax.
"""

# ╔═╡ 15aaa5b6-5e54-11eb-067f-9dcd096940b6
ex1 = Meta.parse(prog)

# ╔═╡ 34b6a71e-5e54-11eb-0324-4bffbe85c813
typeof(ex1)

# ╔═╡ 22770a30-5e54-11eb-28db-1359ab1f402f
md"""
`Expr` objects contains two parts:

  * a `Symbol` identifying the kind of expression. A symbol is an [interned string](https://en.wikipedia.org/wiki/String_interning)
    identifier (string interning is a method of storing only one copy of each distinct string value, which must be immutable).

"""

# ╔═╡ 41f9633c-5e54-11eb-323b-e9703c674b0f
ex1.head

# ╔═╡ 48104478-5e54-11eb-2f23-c98c4ccbb763
md"""
  * the expression arguments, which may be symbols, other expressions, or literal values:
"""

# ╔═╡ 538e2054-5e54-11eb-2b9c-451637e1d8ee
ex1.args

# ╔═╡ 6e3beaf8-5e54-11eb-154b-95f5887efcb0
md"""
Expressions may also be constructed directly in [prefix notation](https://en.wikipedia.org/wiki/Polish_notation) (= Polish notation, a mathematical notation in which operators precede their operands, i.e. `+ 1 1` instead of `1 + 1` with infix notation):
"""

# ╔═╡ 7de376a8-5e54-11eb-0356-9b74225b671e
ex2 = Expr(:call, :+, 1, 1)

# ╔═╡ 825b45a6-5e54-11eb-27e6-f3cca8a2f67e
md"""
The two expressions constructed above – by parsing and by direct construction – are equivalent:
"""

# ╔═╡ 8b23cac8-5e54-11eb-2424-5972ea3d2e21
ex1 == ex2

# ╔═╡ 8ecd49ba-5e54-11eb-24b6-a328043b7a07
md"""
**The key point here is that Julia code is internally represented as a data structure that is accessible
from the language itself.**

The `dump` function provides indented and annotated display of `Expr` objects:
"""

# ╔═╡ 985a57f2-5e54-11eb-1f2f-bf23014c6171
@terminal dump(ex2)

# ╔═╡ d66ea502-5e54-11eb-2694-4766a05aed38
md"""
### Symbols

The `:` character has two syntactic purposes in Julia. The first form creates a [`Symbol`](@ref),
an [interned string](https://en.wikipedia.org/wiki/String_interning) used as one building-block
of expressions:
"""

# ╔═╡ db1e9936-5e54-11eb-376f-8ddea65d976c
s = :foo

# ╔═╡ e4f50fda-5e54-11eb-2210-6b8eb531d898
typeof(s)

# ╔═╡ ea891b6e-5e54-11eb-24c9-357a8a65d1c0
md"""
The `Symbol` constructor takes any number of arguments and creates a new symbol by concatenating
their string representations together:
"""

# ╔═╡ efe79c46-5e54-11eb-23c9-3107ea94e5b9
:foo == Symbol("foo")

# ╔═╡ f5fac50e-5e54-11eb-2ff9-cf7e30a4a8ad
Symbol("func", 10)

# ╔═╡ fe35b31e-5e54-11eb-258e-29913f1e8723
Symbol(:var, '_', "sym")

# ╔═╡ 0464f1be-5e55-11eb-34df-ef1217012663
md"""Note that to use `:` syntax, the symbol's name must be a valid identifier.
Otherwise the `Symbol(str)` constructor must be used.

In the context of an expression, symbols are used to indicate access to variables; when an expression
is evaluated, a symbol is replaced with the value bound to that symbol in the appropriate **scope**.
"""

# ╔═╡ abfc3392-5e5a-11eb-149d-ad3dd2755524
md"""
### Quoting

The second syntactic purpose of the `:` character is to create expression objects without using
the explicit `Expr` constructor. This is referred to as *quoting*. The `:` character, followed
by paired parentheses around a single statement of Julia code, produces an `Expr` object based
on the enclosed code. Here is an example of the short form used to quote an arithmetic expression:
"""

# ╔═╡ 917a579c-5e5a-11eb-3fe6-cb0dcad5e4e6
ex = :(a + b * c + 1)

# ╔═╡ 82d3220a-5e5a-11eb-3c49-d5e7afb1af2a
typeof(ex)

# ╔═╡ 67954702-5e5a-11eb-1ab1-e9a285ee208d
ex.args

# ╔═╡ 2ae6ac9c-5e5a-11eb-060c-b3eae1df0493
md"""
### Interpolation

Direct construction of `Expr` objects with value arguments is powerful, but `Expr` constructors
can be tedious compared to "normal" Julia syntax. As an alternative, Julia allows *interpolation* of
literals or expressions into quoted expressions. Interpolation is indicated by a prefix `$`.

In this example, the value of variable `a` is interpolated:
"""

# ╔═╡ 5ef3d532-5e5a-11eb-30b3-a569e0bd736e
a = 1

# ╔═╡ 3bb1f892-5e5a-11eb-0946-e5e977a4e5b2
:($a + b)

# ╔═╡ 5763cca0-5e5a-11eb-1341-3d9bfb47fb58
md"""
Interpolating into an unquoted expression is not supported and will cause a compile-time error:
"""

# ╔═╡ e6a3902a-5e58-11eb-2bd2-4bc779bcebe7
$a + b

# ╔═╡ 11991248-5e55-11eb-2748-992f6fe48620
md"""
The use of `$` for expression interpolation is intentionally reminiscent of string interpolation and command interpolation. Expression interpolation allows convenient, readable programmatic construction of complex Julia expressions.
"""

# ╔═╡ 1c016bec-5e5d-11eb-3687-931da82f8c04
md"""

### Splatting interpolation

Notice that the `$` interpolation syntax allows inserting only a single expression into an
enclosing expression.
Occasionally, you have an array of expressions and need them all to become arguments of
the surrounding expression.
This can be done with the syntax `$(args...)`.

For example, the following code generates a function call where the number of arguments is
determined programmatically:
"""

# ╔═╡ 552ea3bc-5e5d-11eb-0c97-e1e6db526df3
args = [:x, :y, :z]

# ╔═╡ 629c4f9a-5e5d-11eb-2731-bf7457755829
:(f(1, $(args...)))

# ╔═╡ 66c113d0-5e5d-11eb-0aab-b3fa7f98eae5
md"""
### Evaluating expressions

Given an expression object, one can cause Julia to evaluate (execute) it at global scope using `eval`
"""

# ╔═╡ 6fc2fb9c-5e5d-11eb-0f21-6524555a9c42
:(1 + 2)

# ╔═╡ 6dab1d58-5e5d-11eb-3381-a1796f8d14c5
eval(:(1 + 2))

# ╔═╡ b833d8ba-5e5d-11eb-3311-899efb15c1b0
md"""
## Macros

Now that we have an understanding of the basic concepts of code representation in Julia, we can introduce the core concept of this notebook: macros.
Macros provide a method to include generated code in the final body of a program. A macro maps
a tuple of arguments to a returned *expression*, and the resulting expression is compiled directly
rather than requiring a runtime `eval` call. Macro arguments may include expressions,
literal values, and symbols.

In the following examples, we will show that macros allow us to
1. modify code before it runs
2. elegantly add new features or syntax
3. process strings at compile time instead of runtime

### Basics

Here is an extraordinarily simple macro:
"""

# ╔═╡ 169c6aa2-5e5e-11eb-1d84-e7e3b4641e10
macro sayhello()
	return :( println("Hello, world!") )
end

# ╔═╡ ae1525ea-5e5e-11eb-3906-593143776559
@terminal @sayhello

# ╔═╡ c40cebe6-5e5e-11eb-1f7f-37134b7a449f
md"""
Macros have a dedicated character in Julia's syntax: the `@` (at-sign), followed by the unique
name declared in a `macro NAME ... end` block. In this example, the compiler will replace all
instances of `@sayhello` with:
"""

# ╔═╡ f5ab41f4-5e5f-11eb-02f7-efcb3e05e616
:( println("Hello, world!") )

# ╔═╡ 0001a53a-5e60-11eb-18df-d7e9e7dcf43f
md"""
When `@sayhello` is entered in the REPL, the expression executes immediately, thus we only see the evaluation result:
"""

# ╔═╡ bda7fd66-5e5e-11eb-1e06-eb67a0178e14
@terminal @sayhello()

# ╔═╡ 1707320e-5e60-11eb-3f33-798c81663488
md"""Now, consider a slightly more complex macro:"""

# ╔═╡ 1fdf9600-5e60-11eb-30d3-b167811ae47f
macro sayhello(name)
	return :( println("Hello, ", $name) )
end

# ╔═╡ 2dcacdfc-5e60-11eb-3d04-9f2ffb46d0fe
md"""
This macro takes one argument: `name`. When `@sayhello` is encountered, the quoted expression is *expanded* to interpolate the value of the argument into the final expression:
"""

# ╔═╡ 36fc5076-5e60-11eb-04be-bd9f542b1ff3
@terminal @sayhello("Mr. Bond")

# ╔═╡ 0c787198-61a0-11eb-1e30-fbcb5a624076
@terminal @sayhello "Mr. Bond"

# ╔═╡ 4a732eb8-5e60-11eb-197b-0729d0f1be15
md"""
We can view the quoted return expression using the function `macroexpand` or the macro `@macroexpand` (**important note:**
this is an extremely useful tool for debugging macros).
We can see that the `"Mr. Bond"` literal has been interpolated into the expression."""

# ╔═╡ 536addec-5e61-11eb-1a9d-d7f2c190d6c2
@macroexpand @sayhello("Mr. Bond")

# ╔═╡ 855412b2-5e61-11eb-3aba-bb862489414e
md"""

### Hold on: why macros?

We have already seen a function `f(::Expr...) -> Expr` in a previous section. In fact, `macroexpand` is also such a function. So, why do macros exist?

Macros are necessary because they execute when code is parsed, therefore, macros allow the programmer
to generate and include fragments of customized code *before* the full program is run. To illustrate
the difference, consider the following example.

Note that in this case, you need to look at your REPL to see the output. A macro definition is not allowed in the local scope, so we cannot wrap this with our `@terminal` macro.
"""

# ╔═╡ cabb3cdc-5e62-11eb-2479-bb8337c05292
macro twostep(arg)
	println("I execute at parse time. The argument is: ", arg)
	str1 = "I execute at runtime. "
	str2 = "The argument is: "
	message = str1 * str2
	return :(println($message, $arg))
end

# ╔═╡ f33f0cfe-5f52-11eb-30e0-93c4b899aaa7
md"""Note that the computation of message is compiled away if we expand the macro!"""

# ╔═╡ 01feeca0-5e63-11eb-11a0-1b66a92127d1
ex_twostep = @macroexpand @twostep :(1, 2, 3)

# ╔═╡ f1969888-5e65-11eb-18a3-879d2f87b447
@terminal dump(ex_twostep)

# ╔═╡ 0613f620-5e66-11eb-08d9-01dd3a403321
md"""
### Macro invocation

Macros are invoked with the following general syntax:

```julia
@name expr1 expr2 ...
@name(expr1, expr2, ...)
```

"""

# ╔═╡ 29a2c0ee-5e66-11eb-2194-6f1c3cef563a
md"""
Note the distinguishing `@` before the macro name and the lack of commas between the argument
expressions in the first form, and the lack of whitespace after `@name` in the second form. The
two styles should not be mixed. For example, the following syntax is different from the examples
above; it passes the tuple `(expr1, expr2, ...)` as one argument to the macro:

```julia
@name (expr1, expr2, ...)
```
"""

# ╔═╡ a1e86478-5e66-11eb-2ef9-6d460e707013
md"""
### Building an advanced macro

Here is a simplified definition of Julia's `@assert` macro, which checks if an expression is true:

(`... ? ... : ...` is the ternary if-else operator seen in `01-basics.jl` )
"""

# ╔═╡ e61381aa-5e66-11eb-3347-5d26b61e6c17
macro assert(ex)
	return :( $ex ? nothing : throw(AssertionError($(string(ex)))) )
end

# ╔═╡ a43490de-5e67-11eb-253e-f7455ade3ffe
md"""This macro can be used like this:"""

# ╔═╡ c79ad04c-5e67-11eb-1fd9-bf7ff40f0862
md"""
In place of the written syntax, the macro call is expanded at parse time to its returned result. This is equivalent to writing:"""

# ╔═╡ f34e9020-5e67-11eb-1987-e3e6a0969705
1 == 1.0 ? nothing : throw(AssertionError("1 == 1.0"))

# ╔═╡ f81955fe-5e67-11eb-11b9-5f9fd0096a09
1 == 0 ? nothing : throw(AssertionError("1 == 0"))

# ╔═╡ fb772ef4-5e67-11eb-218e-330ba087bcc8
md"""
That is, in the first call, the expression `:(1 == 1.0)` is spliced into the test condition slot,
while the value of `string(:(1 == 1.0))` is spliced into the assertion message slot. The entire
expression, thus constructed, is placed into the syntax tree where the `@assert` macro call occurs.
Then at execution time, if the test expression evaluates to true, then `nothing` is returned,
whereas if the test is false, an error is raised indicating the asserted expression that was false.
Notice that it would not be possible to write this as a function, since only the *value* of the
condition is available and it would be impossible to display the expression that computed it in
the error message.

The actual definition of `@assert` in Julia Base is more complicated. It allows the
user to optionally specify their own error message, instead of just printing the failed expression.
Just like in functions with a variable number of arguments, this is specified with an ellipses
following the last argument:
"""

# ╔═╡ 2df2baa8-5e68-11eb-2f28-9968b7ecbcd5
macro assert(ex, msgs...)
	msg_body = isempty(msgs) ? ex : msgs[1]
	msg = string(msg_body)
	return :($ex ? nothing : throw(AssertionError($msg)))
end

# ╔═╡ b0dfd0dc-5e67-11eb-19cf-5970a2e80bfa
@assert 1 == 1.0

# ╔═╡ c42d2d6a-5e67-11eb-1564-2dcb0e91fb19
@assert 1 == 0

# ╔═╡ 376fe8e4-5e68-11eb-1d7d-c9b1945b133f
md"""
Now `@assert` has two modes of operation depending upon the number of arguments it receives!
If there is only one argument, the tuple of expressions captured by `msgs` will be empty and it
will behave the same as the simpler definition above with only one argument. But now if the user specifies a second argument,
it is printed in the message body instead of the failing expression. You can inspect the result
of a macro expansion with `@macroexpand`.
"""

# ╔═╡ 4750ce04-5e68-11eb-237e-3fed9eb1f4c5
@macroexpand @assert a == b

# ╔═╡ 4dce26dc-5e68-11eb-1ff7-8974f93e3cb8
@macroexpand @assert a==b "a should equal b!"

# ╔═╡ 558920ca-5e68-11eb-3533-d5d596e3884d
md"""

There is yet another case that the actual `@assert` macro handles: what if, in addition to printing
"a should equal b," we wanted to print their values? One might naively try to use string interpolation
in the custom message, e.g., `@assert a==b "a ($a) should equal b ($b)!"`, but this won't work
as expected with the above macro. Can you see why? Recall from string interpolation that an interpolated string is rewritten to a call to `string`. Compare:
"""

# ╔═╡ 6d69106a-5e68-11eb-0d50-e17b4db5db74
typeof(:("a should equal b"))

# ╔═╡ 715464cc-5e68-11eb-3e1a-977db04a7db6
typeof(:("a ($a) should equal b ($b)!"))

# ╔═╡ 7588018e-5e68-11eb-3be2-b97ce124d6bf
@terminal dump(:("a ($a) should equal b ($b)!"))

# ╔═╡ 84771eaa-5e68-11eb-0684-cf1a3118d10b
md"""
So now instead of getting a plain string in `msg_body`, the macro is receiving a full expression that will need to be evaluated in order to display as expected. This can be spliced directly into the returned expression as an argument to the `string` call; see `error.jl` for the complete implementation.

The `@assert` macro makes great use of splicing into quoted expressions to simplify the manipulation of expressions inside the macro body.
"""

# ╔═╡ 43a19ace-5e6b-11eb-005b-b7d2d9a46878
md"""
### Macros and dispatch

Macros, just like Julia functions, are generic. This means they can also have multiple method definitions, thanks to multiple dispatch:
"""

# ╔═╡ 508167c4-5e6b-11eb-2587-b967b79cf74c
macro m end

# ╔═╡ 59f86d98-5e6b-11eb-3259-7312501b70bf
macro m(args...)
	"$(length(args)) arguments"
end

# ╔═╡ 5edf55e2-5e6b-11eb-383e-5965ae694c4c
macro m(x,y)
   	"Two arguments"
end

# ╔═╡ 3dc3aa10-5e6c-11eb-3338-7777b8cb97a5
md"""
However one should keep in mind, that macro dispatch is based on the types of AST
that are handed to the macro, not the types that the AST evaluates to at runtime:"""

# ╔═╡ 34565ba8-5e6c-11eb-2a27-2ff13394bcc8
macro m(::Int)
	"An Integer"
end

# ╔═╡ 65c94e9c-5e6b-11eb-38a8-4d60c692222b
@m "asdl"

# ╔═╡ 42fc5284-5e6c-11eb-0edb-9555a320ec55
@m 1 2

# ╔═╡ 02e4bf64-5e6d-11eb-36b5-fb57c87bff81
@m 3

# ╔═╡ 07f8cc34-5e6d-11eb-36e4-b56b2b92a2e2
x = 2

# ╔═╡ 0bbfce80-5e6d-11eb-07f6-b71086357384
@m x

# ╔═╡ 0e00bff6-5e6d-11eb-342d-bfded7ef8a96
md"""
## Code Generation

When a significant amount of repetitive boilerplate code is required, it is common to generate
it programmatically to avoid redundancy. In most languages, this requires an extra build step,
and a separate program to generate the repetitive code. In Julia, expression interpolation and `eval` allow such code generation to take place in the normal course of program execution.
For example, consider the following custom type
"""

# ╔═╡ 36805374-5e6d-11eb-205c-0524096bdf27
struct MyNumber
    x::Float64
end

# ╔═╡ 4056c9bc-5e6d-11eb-377e-238ef8633c74
md"""for which we want to add a number of methods to. We can do this programmatically in the following loop:"""

# ╔═╡ 49807d8c-5e6d-11eb-0170-f1419837d958
for op = (:sin, :cos, :tan, :log, :exp, :log)
    @eval Base.$op(a::MyNumber) = MyNumber($op(a.x))
end

# ╔═╡ 4ef3f280-5e6d-11eb-2020-b7cd9e6b9c03
md"""and we can now use those functions with our custom type:"""

# ╔═╡ 53f36f36-5e6d-11eb-3f06-6de0eff4d173
y = MyNumber(π)

# ╔═╡ 3428d7b8-5e6d-11eb-32bc-af1df0579e60
sin(y)

# ╔═╡ 18e3753a-5f55-11eb-0a11-47ca6abe9186
md"This will not work since we only have defined `log` and not `log10`."

# ╔═╡ 2f4fe6ce-5e6c-11eb-1744-2f9534c8b6ba
log10(y)

# ╔═╡ cae2546a-5e6b-11eb-3fc2-69f9a15107cf
md"""In this manner, Julia acts as its own preprocessor, and allows code generation from inside the language."""

# ╔═╡ 0ac5096e-61a1-11eb-24ba-13bc0a3377c6

# ╔═╡ efc1357a-61a0-11eb-20f5-a15338080e4c
md"### Example: domain specific languages"

# ╔═╡ 4d9de330-5f55-11eb-0d89-eb81e6c9ffab
md"
Code generation can for instance be used to simplify the creation of a mathematical optimisation problem. In this case, we we'll use the `JuMP` package.
`JuMP` ('Julia for Mathematical Programming') is an open-source modeling language that is embedded in Julia. It allows users to formulate various classes of optimization problems (linear, mixed-integer, quadratic, conic quadratic, semidefinite, and nonlinear) with easy-to-read code. `JuMP` also makes advanced optimization techniques easily accessible from a high-level language.

As a dummy example, let us consider the following linear programming problem:

``\max_{x,y}\,\,x + 2y``

``\text{s.t.}``

``x + y \leq 1``

``0\leq x, y \leq 1``

Which can be transcribed into a `JuMP` model as:
"

# ╔═╡ 92458f54-5f57-11eb-179f-c749ce06f7e2
let
	model = Model(GLPK.Optimizer)
	@variable(model, 0 <= x <= 1)
	@variable(model, 0 <= y <= 1)
	@constraint(model, x + y <= 1)
	@objective(model, Max, x + 2y)
	optimize!(model)
	value(x), value(y), objective_value(model)
end

# ╔═╡ d1222dcc-5f57-11eb-2c87-77a16ea8e65d
md"
Without the macros, the code would be more difficult to read. As an example, check the `macroexpansion` of `@constraint` to see the bunch of code that is generated behind the scenes:
"

# ╔═╡ 0b1f3542-5f58-11eb-1610-75df3e2a4a76
let
	model = Model(GLPK.Optimizer)
	@macroexpand @constraint(model, x + y <= 1)
end

# ╔═╡ 57f243ec-5fe3-11eb-04d9-d776ccba4c0f
md"Finally, this section on code generation ends with two more examples:
1. a macro to repeat a certain expression n-times
2. a macro to repeat a certain expression until a condition is met

Are these macros the best way to tackle the problems at hand? Maybe not, but they do give a nice illustration of how the code generation works.
"

# ╔═╡ 211cf9ca-5fe3-11eb-2596-71b1848753d7
macro dotimes(n, body)
    quote
        for i = 1:$(esc(n))
            $(esc(body))
        end
    end
end

# ╔═╡ 35ccaee2-5fe3-11eb-3bd0-e355b0fd3f71
@macroexpand @dotimes 3 println("hi there")

# ╔═╡ 434669d2-5fe3-11eb-1f4a-27edf1f9e7f6
macro until(condition, block)
    quote
        while true
            $(esc(block))
            if $(esc(condition))
                break
            end
        end
    end
end

# ╔═╡ 9612965e-5fe3-11eb-172a-59cc73a26ab6
PlutoUI.with_terminal() do
	i = 0
	@until i == 10 begin
		i += 1
		println(i)
	end
end

# ╔═╡ 3c2cce20-5fe3-11eb-32e3-f542f9a94a6e
@macroexpand @until j == 10 begin
	j += 1
	println(j)
end

# ╔═╡ 10a8532a-5e60-11eb-3331-974e708cb39d
md"""
## Non-Standard String Literals

String literals prefixed by an identifier are called non-standard string literals, and can have different semantics than un-prefixed string literals. For example:

  * `r"^\s*(?:#|$)"` produces a regular expression object rather than a string
  * `b"DATA\xff\u2200"` is a byte array literal for `[68,65,84,65,255,226,136,128]`.

Perhaps surprisingly, these behaviors are not hard-coded into the Julia parser or compiler. Instead,
they are custom behaviors provided by a general mechanism that anyone can use: prefixed string
literals are parsed as calls to specially-named macros. For example, the regular expression macro
is just the following:
```julia
macro r_str(p)
    Regex(p)
end
```
"""

# ╔═╡ d8a9ad46-5e7a-11eb-1cce-3d4bc49cd332
md"""
That's all. This macro says that the literal contents of the string literal `r"^\s*(?:#|$)"` should
be passed to the `@r_str` macro and the result of that expansion should be placed in the syntax
tree where the string literal occurs. In other words, the expression `r"^\s*(?:#|$)"` is equivalent
to placing the following object directly into the syntax tree:
"""

# ╔═╡ e85b2d3c-5e7a-11eb-3b89-e11bc274f7cf
Regex("^\\s*(?:#|\$)")

# ╔═╡ ec73cd70-5e7a-11eb-0285-6b1fabd1289d
md"""
Not only is the string literal form shorter and far more convenient, but it is also more efficient:
since the regular expression is compiled, which takes time, and the `Regex` object is actually created *when the code is compiled*,
the compilation occurs only once, rather than every time the code is executed. Consider if the
regular expression occurs in a loop:
"""

# ╔═╡ f1d24ba2-5e7a-11eb-010a-f543cfc306b5
PlutoUI.with_terminal() do
	for line ∈ ["first", "sec#nd", "third"]
		m = match(r"#", line)
		if m === nothing
			println("nothing found")
		else
			println("found something")
		end
	end
end

# ╔═╡ f838762c-5e7a-11eb-000a-e963fab5e97d
md"""
Since the regular expression `r"^\s*(?:#|$)"` is compiled and inserted into the syntax tree when
this code is parsed, the expression is only compiled once instead of each time the loop is executed.
In order to accomplish this without macros, one would have to write this loop like this:
"""

# ╔═╡ 149c1c12-5e7b-11eb-297d-6da57f45891b
PlutoUI.with_terminal() do
	re = Regex("#")
	for line ∈ ["first", "sec#nd", "third"]
		m = match(re, line)
		if m === nothing
			println("nothing found")
		else
			println("found something")
		end
	end
end

# ╔═╡ 780da60e-5e7c-11eb-0c1a-55be73da188f
md"""
Moreover, if the compiler could not determine that the regex object was constant over all loops,
certain optimizations might not be possible, making this version still less efficient than the
more convenient literal form above. Of course, there are still situations where the non-literal
form is more convenient: if one needs to interpolate a variable into the regular expression, one
must take this more verbose approach. In the vast majority of use cases, however, regular expressions
are not constructed based on run-time data. In this majority of cases, the ability to write regular
expressions as compile-time values is invaluable.
"""

# ╔═╡ 90a672d0-5f5c-11eb-3a62-1b034cd41e67
md"
Designing your own custom string literals can be done as such:
"

# ╔═╡ b4104d12-5e7c-11eb-3e22-a78b74526129
macro foo_str(str, flag)
    # do stuff
	str, flag
end

# ╔═╡ 20393e86-5e7d-11eb-18e3-613890472903
foo"this is the string"theflag

# ╔═╡ 5069df16-5e7d-11eb-217d-6b740e9b3559
md"""
The first example of a custom macro can be found in every notebook in this course! It is the markdown string literal, which allows the usage of Markdown markup language to prettify these lectures!

```julia
md"I am a Markdown string with glorious **formatting capabilities**."
```
I am a Markdown string with glorious *formatting* **capabilities**.
"""

# ╔═╡ 1a50ff0e-5e7d-11eb-2fc4-cd5c12015751
md"The second example is from the `BioSequences.jl` package. In that bioinformatics package, you can define sequences e.g. DNA and RNA as string literals:
"

# ╔═╡ 0664eb22-5e7d-11eb-07a6-ed35143bf03f
dna"ACGT"

# ╔═╡ a965bd70-5e7c-11eb-13dd-5fe83950af11
md"Repetition and concatenation can be performed pretty straightforward:"

# ╔═╡ fb0c59fa-5f61-11eb-1ab7-bf7b80746aa2
repeat(dna"TTAGGG", 10)

# ╔═╡ a157dafc-5e7c-11eb-105d-4db3ee9dbec6
dna = dna"ACGT" * dna"TGCAA"

# ╔═╡ 0ec9887c-5f5e-11eb-29e5-931881dcb222
md"Other typical string operations such as pushing new values work as you would expect"

# ╔═╡ 1d4aca5a-5f5e-11eb-23c3-d1ddef4e6ee6
push!(dna, DNA_A)

# ╔═╡ c4c2137a-5f5d-11eb-1e06-8f5e2b1f1c0e
md"There exist methods to convert a DNA sequence to its RNA equivalent:"

# ╔═╡ e13fcbbe-5f5d-11eb-2074-fdbc945861e1
rna = convert(LongRNASeq, dna)

# ╔═╡ 2c1308f4-5f5e-11eb-058f-79cbf7dd23c6
md"Note that altough the printout of the DNA and RNA object is different because of different nucleotides. The information content is the same, and as such this statement is true:"

# ╔═╡ fdf46c92-5f5d-11eb-345b-e145c9c2874c
dna.data === rna.data

# ╔═╡ 37da420a-5f61-11eb-3688-09e6188165e3
md"Remember before that we mentioned that you can add a flag to a string literal? In `BioSequences.jl` this has a use case.

If you have a function that generates a sequence, and you want it to create a new sequence each time it is called, then you can add a flag to the end of the sequence literal to dictate behaviour: A flag of 's' means 'static': the sequence will be allocated before code is run, as is the default behaviour. However providing 'd' flag changes the behaviour: 'd' means 'dynamic': the sequence will be allocated whilst the code is running, and not before. So to change foo so as it creates a new sequence each time it is called, simply add the 'd' flag to the sequence literal:"

# ╔═╡ d9f18616-5e8d-11eb-1b1a-3bcc749fb467
function getdna_dynamic()
	s = dna"CTT"d     # 'd' flag appended to the string literal.
	push!(s, DNA_A)
end

# ╔═╡ ea3da4dc-5e8d-11eb-0910-091b926aea58
getdna_dynamic()

# ╔═╡ f13dd22a-5e8d-11eb-1def-137de66123ba
getdna_dynamic()

# ╔═╡ be11ec9a-5f61-11eb-374a-c96489ea582d
md"Output of `getdna_dynamic()` stays the same!"

# ╔═╡ f44d757e-5e8d-11eb-18f1-719d3c7b8688
function getdna_static()
	s = dna"CTT"s     # 's' flag appended to the string literal.
	push!(s, DNA_A)
end

# ╔═╡ fa310fdc-5e8d-11eb-024b-7ded3e213112
getdna_static()

# ╔═╡ 1acb02d4-5e8e-11eb-232c-f756e92c3f97
getdna_static()

# ╔═╡ 2dcb4600-5e8e-11eb-093e-59eb79f0cf20
md"""
Be careful when you are using sequence literals inside of functions, and inside the bodies of things like for loops. And if you use them and are unsure, use the 's' and 'd' flags to ensure the behaviour you get is the behaviour you intend.
"""

# ╔═╡ 2e2601b6-5e94-11eb-3613-eb5fee19b6b7
md"
## Overview of some interesting macros
"

# ╔═╡ 311a4666-5fc5-11eb-2cc0-ef66e5be96e3
md"check if an expression is true"

# ╔═╡ 4622422a-5fc5-11eb-20f2-9bfe466f8f30
@assert true == true

# ╔═╡ 5ba5d6b6-5fc5-11eb-256b-73a954a5db68
md"Integers and floating point numbers with arbitrary precision. This macro exists because promoting a floating point number to a `BigFloat` will keep the approximation error of `Float64`."

# ╔═╡ a47ab60e-5fc5-11eb-368a-d76fc6ee640d
big"0.1"

# ╔═╡ a99546c2-5fc5-11eb-0902-71cb283543c8
@big_str "0.1"

# ╔═╡ af2c7d4e-5fc5-11eb-30d4-a367d2c0db23
big(0.1)

# ╔═╡ 4a335466-5fc6-11eb-2273-312fd3928616
md"There exists a lot more unique string literals than we have shown here, such as html strings, ip address literals, etc."

# ╔═╡ 17f771dc-5fc6-11eb-3b83-ed51f87194e0
html"""<!-- HTML generated using hilite.me --><div style="background: #ffffff; overflow:auto;width:auto;border:solid gray;border-width:.1em .1em .1em .8em;padding:.2em .6em;"><table><tr><td><pre style="margin: 0; line-height: 125%">1</pre></td><td><pre style="margin: 0; line-height: 125%">println(<span style="color: #a31515">"hello world!"</span>)
</pre></td></tr></table></div>
"""

# ╔═╡ 92836b48-5fc6-11eb-19b6-1d7a5c310052
@terminal println("There is nothing like ", ip"127.0.0.1")

# ╔═╡ f26e65a8-5fc6-11eb-3cd7-3f40c12a6abb
md"Find out what function is exactly used in multiple dipatching."

# ╔═╡ 154e2392-5fc7-11eb-250b-e726ab7710e7
@which sin(2.2)

# ╔═╡ 2cfa3952-5fc7-11eb-2e79-95f86aafbf2c
@which sin(2)

# ╔═╡ 4aece42a-5fc7-11eb-291f-99120483339f
md"
There exist some handy macros to analyse execution time and memory allocation.

```julia
using BenchmarkTools
@btime sin(2)
@benchmark sin(2)
@elapsed sin(2)
```
"

# ╔═╡ 295e0028-5fd6-11eb-39e4-b392de1c0de2
md"""
When developing modules, scripts or julia packages, you can use the `@info`, `@warn`, `@error` and `@debug` as logging macros. They are mostly useful in packages, not in notebooks like this.

An example usage would be a warning thrown by an optimisation algorithm to tell you that e.g. the predefined accuracy or tolerance was not reached.

Note, to make `@debug` you need to set an environment variable to mark that you are in debug mode: `ENV["JULIA_DEBUG"] = "all"`
"""

# ╔═╡ 4a9d85ba-5fc7-11eb-1b6e-59ef2259dd24
PlutoUI.with_terminal() do
	@info "Information comes here"
	@error "Error has been found at this exact location"
	@warn "Same. but for a warning"
end

# ╔═╡ 34829ada-5fdc-11eb-263a-cf60a9650556
md"Plot recipes are a nifty thing to make plots for your custom data type:"

# ╔═╡ 5c33798c-5fda-11eb-186f-79450fb59e3b
struct TemperatureMeas
	t1
	t2
	t3
	TemperatureMeas(t1, t2, t3) = new(t1, t2, t3)
end

# ╔═╡ 1ca80d42-5fdb-11eb-153f-9ddb25a4fb6b
@recipe function f(tempmeas::TemperatureMeas)
    xguide --> "time (seconds)"
    yguide --> "temperature (Celsius)"
    [tempmeas.t1, tempmeas.t2, tempmeas.t3] # return the arguments (input data) for the next recipe
end

# ╔═╡ 1a793992-5fdb-11eb-3b38-05544f268a65
plot(TemperatureMeas(Plots.fakedata(50), Plots.fakedata(50), Plots.fakedata(50)))

# ╔═╡ d418dca4-5fe5-11eb-3f1a-a31e5dc5f7e9
md"
Using the `Distributed` module, one can easily transform certain types of code to run distributed (i.e. on multiple cores). Below is an example of how you can transform a for-loop into its distributed version.
"

# ╔═╡ f1b70998-5fdb-11eb-292b-0776f1b03816
PlutoUI.with_terminal() do
	@sync @distributed for i ∈ 1:5
		println(i)
	end
end

# ╔═╡ Cell order:
# ╠═42e620aa-5f4c-11eb-2ebf-85814cf720e7
# ╟─31c1e25e-5e53-11eb-2467-9153d30962d5
# ╟─4ab33c0e-5e53-11eb-2e63-2dd6f06de3ba
# ╠═e76d8f04-5e53-11eb-26df-db496622642d
# ╟─f361d734-5e53-11eb-3957-61bd0a370b5a
# ╠═15aaa5b6-5e54-11eb-067f-9dcd096940b6
# ╠═34b6a71e-5e54-11eb-0324-4bffbe85c813
# ╟─22770a30-5e54-11eb-28db-1359ab1f402f
# ╠═41f9633c-5e54-11eb-323b-e9703c674b0f
# ╟─48104478-5e54-11eb-2f23-c98c4ccbb763
# ╠═538e2054-5e54-11eb-2b9c-451637e1d8ee
# ╟─6e3beaf8-5e54-11eb-154b-95f5887efcb0
# ╠═7de376a8-5e54-11eb-0356-9b74225b671e
# ╟─825b45a6-5e54-11eb-27e6-f3cca8a2f67e
# ╠═8b23cac8-5e54-11eb-2424-5972ea3d2e21
# ╟─8ecd49ba-5e54-11eb-24b6-a328043b7a07
# ╠═985a57f2-5e54-11eb-1f2f-bf23014c6171
# ╟─d66ea502-5e54-11eb-2694-4766a05aed38
# ╠═db1e9936-5e54-11eb-376f-8ddea65d976c
# ╠═e4f50fda-5e54-11eb-2210-6b8eb531d898
# ╟─ea891b6e-5e54-11eb-24c9-357a8a65d1c0
# ╠═efe79c46-5e54-11eb-23c9-3107ea94e5b9
# ╠═f5fac50e-5e54-11eb-2ff9-cf7e30a4a8ad
# ╠═fe35b31e-5e54-11eb-258e-29913f1e8723
# ╟─0464f1be-5e55-11eb-34df-ef1217012663
# ╟─abfc3392-5e5a-11eb-149d-ad3dd2755524
# ╠═917a579c-5e5a-11eb-3fe6-cb0dcad5e4e6
# ╠═82d3220a-5e5a-11eb-3c49-d5e7afb1af2a
# ╠═67954702-5e5a-11eb-1ab1-e9a285ee208d
# ╟─2ae6ac9c-5e5a-11eb-060c-b3eae1df0493
# ╠═5ef3d532-5e5a-11eb-30b3-a569e0bd736e
# ╠═3bb1f892-5e5a-11eb-0946-e5e977a4e5b2
# ╟─5763cca0-5e5a-11eb-1341-3d9bfb47fb58
# ╠═e6a3902a-5e58-11eb-2bd2-4bc779bcebe7
# ╟─11991248-5e55-11eb-2748-992f6fe48620
# ╟─1c016bec-5e5d-11eb-3687-931da82f8c04
# ╠═552ea3bc-5e5d-11eb-0c97-e1e6db526df3
# ╠═629c4f9a-5e5d-11eb-2731-bf7457755829
# ╟─66c113d0-5e5d-11eb-0aab-b3fa7f98eae5
# ╠═6fc2fb9c-5e5d-11eb-0f21-6524555a9c42
# ╠═6dab1d58-5e5d-11eb-3381-a1796f8d14c5
# ╟─b833d8ba-5e5d-11eb-3311-899efb15c1b0
# ╠═169c6aa2-5e5e-11eb-1d84-e7e3b4641e10
# ╠═ae1525ea-5e5e-11eb-3906-593143776559
# ╟─c40cebe6-5e5e-11eb-1f7f-37134b7a449f
# ╠═f5ab41f4-5e5f-11eb-02f7-efcb3e05e616
# ╟─0001a53a-5e60-11eb-18df-d7e9e7dcf43f
# ╠═bda7fd66-5e5e-11eb-1e06-eb67a0178e14
# ╟─1707320e-5e60-11eb-3f33-798c81663488
# ╠═1fdf9600-5e60-11eb-30d3-b167811ae47f
# ╟─2dcacdfc-5e60-11eb-3d04-9f2ffb46d0fe
# ╠═36fc5076-5e60-11eb-04be-bd9f542b1ff3
# ╠═0c787198-61a0-11eb-1e30-fbcb5a624076
# ╟─4a732eb8-5e60-11eb-197b-0729d0f1be15
# ╠═536addec-5e61-11eb-1a9d-d7f2c190d6c2
# ╟─855412b2-5e61-11eb-3aba-bb862489414e
# ╠═cabb3cdc-5e62-11eb-2479-bb8337c05292
# ╟─f33f0cfe-5f52-11eb-30e0-93c4b899aaa7
# ╠═01feeca0-5e63-11eb-11a0-1b66a92127d1
# ╠═f1969888-5e65-11eb-18a3-879d2f87b447
# ╟─0613f620-5e66-11eb-08d9-01dd3a403321
# ╟─29a2c0ee-5e66-11eb-2194-6f1c3cef563a
# ╟─a1e86478-5e66-11eb-2ef9-6d460e707013
# ╠═e61381aa-5e66-11eb-3347-5d26b61e6c17
# ╟─a43490de-5e67-11eb-253e-f7455ade3ffe
# ╠═b0dfd0dc-5e67-11eb-19cf-5970a2e80bfa
# ╠═c42d2d6a-5e67-11eb-1564-2dcb0e91fb19
# ╟─c79ad04c-5e67-11eb-1fd9-bf7ff40f0862
# ╠═f34e9020-5e67-11eb-1987-e3e6a0969705
# ╠═f81955fe-5e67-11eb-11b9-5f9fd0096a09
# ╟─fb772ef4-5e67-11eb-218e-330ba087bcc8
# ╠═2df2baa8-5e68-11eb-2f28-9968b7ecbcd5
# ╟─376fe8e4-5e68-11eb-1d7d-c9b1945b133f
# ╠═4750ce04-5e68-11eb-237e-3fed9eb1f4c5
# ╠═4dce26dc-5e68-11eb-1ff7-8974f93e3cb8
# ╟─558920ca-5e68-11eb-3533-d5d596e3884d
# ╠═6d69106a-5e68-11eb-0d50-e17b4db5db74
# ╠═715464cc-5e68-11eb-3e1a-977db04a7db6
# ╠═7588018e-5e68-11eb-3be2-b97ce124d6bf
# ╟─84771eaa-5e68-11eb-0684-cf1a3118d10b
# ╟─43a19ace-5e6b-11eb-005b-b7d2d9a46878
# ╠═508167c4-5e6b-11eb-2587-b967b79cf74c
# ╠═59f86d98-5e6b-11eb-3259-7312501b70bf
# ╠═5edf55e2-5e6b-11eb-383e-5965ae694c4c
# ╠═65c94e9c-5e6b-11eb-38a8-4d60c692222b
# ╠═42fc5284-5e6c-11eb-0edb-9555a320ec55
# ╟─3dc3aa10-5e6c-11eb-3338-7777b8cb97a5
# ╠═34565ba8-5e6c-11eb-2a27-2ff13394bcc8
# ╠═02e4bf64-5e6d-11eb-36b5-fb57c87bff81
# ╠═07f8cc34-5e6d-11eb-36e4-b56b2b92a2e2
# ╠═0bbfce80-5e6d-11eb-07f6-b71086357384
# ╟─0e00bff6-5e6d-11eb-342d-bfded7ef8a96
# ╠═36805374-5e6d-11eb-205c-0524096bdf27
# ╟─4056c9bc-5e6d-11eb-377e-238ef8633c74
# ╠═49807d8c-5e6d-11eb-0170-f1419837d958
# ╟─4ef3f280-5e6d-11eb-2020-b7cd9e6b9c03
# ╠═53f36f36-5e6d-11eb-3f06-6de0eff4d173
# ╠═3428d7b8-5e6d-11eb-32bc-af1df0579e60
# ╟─18e3753a-5f55-11eb-0a11-47ca6abe9186
# ╠═2f4fe6ce-5e6c-11eb-1744-2f9534c8b6ba
# ╟─cae2546a-5e6b-11eb-3fc2-69f9a15107cf
# ╟─0ac5096e-61a1-11eb-24ba-13bc0a3377c6
# ╟─efc1357a-61a0-11eb-20f5-a15338080e4c
# ╟─4d9de330-5f55-11eb-0d89-eb81e6c9ffab
# ╠═122cffca-5fdc-11eb-3555-b39b818f1116
# ╠═92458f54-5f57-11eb-179f-c749ce06f7e2
# ╟─d1222dcc-5f57-11eb-2c87-77a16ea8e65d
# ╠═0b1f3542-5f58-11eb-1610-75df3e2a4a76
# ╟─57f243ec-5fe3-11eb-04d9-d776ccba4c0f
# ╠═211cf9ca-5fe3-11eb-2596-71b1848753d7
# ╠═35ccaee2-5fe3-11eb-3bd0-e355b0fd3f71
# ╠═434669d2-5fe3-11eb-1f4a-27edf1f9e7f6
# ╠═9612965e-5fe3-11eb-172a-59cc73a26ab6
# ╠═3c2cce20-5fe3-11eb-32e3-f542f9a94a6e
# ╟─10a8532a-5e60-11eb-3331-974e708cb39d
# ╟─d8a9ad46-5e7a-11eb-1cce-3d4bc49cd332
# ╠═e85b2d3c-5e7a-11eb-3b89-e11bc274f7cf
# ╟─ec73cd70-5e7a-11eb-0285-6b1fabd1289d
# ╠═f1d24ba2-5e7a-11eb-010a-f543cfc306b5
# ╟─f838762c-5e7a-11eb-000a-e963fab5e97d
# ╠═149c1c12-5e7b-11eb-297d-6da57f45891b
# ╟─780da60e-5e7c-11eb-0c1a-55be73da188f
# ╟─90a672d0-5f5c-11eb-3a62-1b034cd41e67
# ╠═b4104d12-5e7c-11eb-3e22-a78b74526129
# ╠═20393e86-5e7d-11eb-18e3-613890472903
# ╟─5069df16-5e7d-11eb-217d-6b740e9b3559
# ╟─1a50ff0e-5e7d-11eb-2fc4-cd5c12015751
# ╠═0963185c-5fdc-11eb-0eed-89d514850353
# ╠═0664eb22-5e7d-11eb-07a6-ed35143bf03f
# ╟─a965bd70-5e7c-11eb-13dd-5fe83950af11
# ╠═fb0c59fa-5f61-11eb-1ab7-bf7b80746aa2
# ╠═a157dafc-5e7c-11eb-105d-4db3ee9dbec6
# ╟─0ec9887c-5f5e-11eb-29e5-931881dcb222
# ╠═1d4aca5a-5f5e-11eb-23c3-d1ddef4e6ee6
# ╟─c4c2137a-5f5d-11eb-1e06-8f5e2b1f1c0e
# ╠═e13fcbbe-5f5d-11eb-2074-fdbc945861e1
# ╟─2c1308f4-5f5e-11eb-058f-79cbf7dd23c6
# ╠═fdf46c92-5f5d-11eb-345b-e145c9c2874c
# ╟─37da420a-5f61-11eb-3688-09e6188165e3
# ╠═d9f18616-5e8d-11eb-1b1a-3bcc749fb467
# ╠═ea3da4dc-5e8d-11eb-0910-091b926aea58
# ╠═f13dd22a-5e8d-11eb-1def-137de66123ba
# ╟─be11ec9a-5f61-11eb-374a-c96489ea582d
# ╠═f44d757e-5e8d-11eb-18f1-719d3c7b8688
# ╠═fa310fdc-5e8d-11eb-024b-7ded3e213112
# ╠═1acb02d4-5e8e-11eb-232c-f756e92c3f97
# ╟─2dcb4600-5e8e-11eb-093e-59eb79f0cf20
# ╟─2e2601b6-5e94-11eb-3613-eb5fee19b6b7
# ╟─311a4666-5fc5-11eb-2cc0-ef66e5be96e3
# ╠═4622422a-5fc5-11eb-20f2-9bfe466f8f30
# ╟─5ba5d6b6-5fc5-11eb-256b-73a954a5db68
# ╠═a47ab60e-5fc5-11eb-368a-d76fc6ee640d
# ╠═a99546c2-5fc5-11eb-0902-71cb283543c8
# ╠═af2c7d4e-5fc5-11eb-30d4-a367d2c0db23
# ╟─4a335466-5fc6-11eb-2273-312fd3928616
# ╟─17f771dc-5fc6-11eb-3b83-ed51f87194e0
# ╠═24806108-5fdc-11eb-2f19-bb09f836f893
# ╠═92836b48-5fc6-11eb-19b6-1d7a5c310052
# ╟─f26e65a8-5fc6-11eb-3cd7-3f40c12a6abb
# ╠═154e2392-5fc7-11eb-250b-e726ab7710e7
# ╠═2cfa3952-5fc7-11eb-2e79-95f86aafbf2c
# ╟─4aece42a-5fc7-11eb-291f-99120483339f
# ╟─295e0028-5fd6-11eb-39e4-b392de1c0de2
# ╠═4a9d85ba-5fc7-11eb-1b6e-59ef2259dd24
# ╟─34829ada-5fdc-11eb-263a-cf60a9650556
# ╠═4621c212-5fc7-11eb-2c9d-ad577506420e
# ╠═5c33798c-5fda-11eb-186f-79450fb59e3b
# ╠═1ca80d42-5fdb-11eb-153f-9ddb25a4fb6b
# ╠═1a793992-5fdb-11eb-3b38-05544f268a65
# ╟─d418dca4-5fe5-11eb-3f1a-a31e5dc5f7e9
# ╠═b2c1cef8-5fe5-11eb-20c7-134432196893
# ╠═f1b70998-5fdb-11eb-292b-0776f1b03816


## Exercise 4-3
## Write a function call that passes t as an argument to square, and then run the macro again.
using ThinkJulia

println("Ans: ")

function square(turtle::Turtle, distance::Int = 100)
    @svg begin
        for i in 1:3
            forward(turtle, distance)
            turn(turtle, -90)
        end

        forward(turtle, distance)
    end
end

function drawSquare()
    turtle = Turtle()
    square(turtle)
end

drawSquare()

println("End.")


@inline function _to_cartesian(a, i::CanonicalInt)
    @inbounds(CartesianIndices(ntuple(dim -> indices(a, dim), Val(ndims(a))))[i])
end
@inline function _to_linear(a, i::Tuple{CanonicalInt,Vararg{CanonicalInt}})
    _strides2int(offsets(a), size_to_strides(size(a), static(1)), i) + static(1)
end

"""
    is_splat_index(::Type{T}) -> StaticBool

Returns `static(true)` if `T` is a type that splats across multiple dimensions.
"""
is_splat_index(@nospecialize(x)) = is_splat_index(typeof(x))
is_splat_index(::Type{T}) where {T} = static(false)
_is_splat(::Type{I}, i::StaticInt) where {I} = is_splat_index(_get_tuple(I, i))

"""
    ndims_index(::Type{I}) -> StaticInt

Returns the number of dimension that an instance of `I` maps to when indexing. For example,
`CartesianIndex{3}` maps to 3 dimensions. If this method is not explicitly defined, then `1`
is returned.

"""
ndims_index(@nospecialize(i)) = ndims_index(typeof(i))
ndims_index(::Type{I}) where {I} = static(1)
ndims_index(::Type{<:AbstractCartesianIndex{N}}) where {N} = static(N)
ndims_index(::Type{<:AbstractArray{T}}) where {T} = ndims_index(T)
ndims_index(::Type{<:AbstractArray{Bool,N}}) where {N} = static(N)
ndims_index(::Type{<:LogicalIndex{<:Any,<:AbstractArray{Bool,N}}}) where {N} = static(N)
_ndims_index(::Type{I}, i::StaticInt) where {I} = ndims_index(_get_tuple(I, i))

"""
    to_indices(A, I::Tuple) -> Tuple

Converts the tuple of indexing arguments, `I`, into an appropriate form for indexing into `A`.
Typically, each index should be an `Int`, `StaticInt`, a collection with values of `Int`, or a collection with values of `CartesianIndex`
This is accomplished in three steps after the initial call to `to_indices`:

# Extended help

This implementation differs from that of `Base.to_indices` in the following ways:

*  `to_indices(A, I)` never results in recursive processing of `I` through
  `to_indices(A, axes(A), I)`. This is avoided through the use of an internal `@generated`
  method that aligns calls of `to_indices` and `to_index` based on the return values of
  `ndims_index`. This is beneficial because the compiler currently does not optimize away
  the increased time spent recursing through
    each additional argument that needs converting. For example:
    ```julia
    julia> x = rand(4,4,4,4,4,4,4,4,4,4);

    julia> inds1 = (1, 2, 1, 2, 1, 2, 1, 2, 1, 2);

    julia> inds2 = (1, CartesianIndex(1, 2), 1, CartesianIndex(1, 2), 1, CartesianIndex(1, 2), 1);

    julia> inds3 = (fill(true, 4, 4), 2, fill(true, 4, 4), 2, 1, fill(true, 4, 4), 1);

    julia> @btime Base.to_indices(\$x, \$inds2)
    1.105 μs (12 allocations: 672 bytes)
    (1, 1, 2, 1, 1, 2, 1, 1, 2, 1)

    julia> @btime ArrayInterface.to_indices(\$x, \$inds2)
    0.041 ns (0 allocations: 0 bytes)
    (1, 1, 2, 1, 1, 2, 1, 1, 2, 1)

    julia> @btime Base.to_indices(\$x, \$inds3);
    340.629 ns (14 allocations: 768 bytes)

    julia> @btime ArrayInterface.to_indices(\$x, \$inds3);
    11.614 ns (0 allocations: 0 bytes)

    ```
* Recursing through `to_indices(A, axes, I::Tuple{I1,Vararg{Any}})` is intended to provide
  context for processing `I1`. However, this doesn't tell use how many dimensions are
  consumed by what is in `Vararg{Any}`. Using `ndims_index` to directly align the axes of
  `A` with each value in `I` ensures that a `CartesiaIndex{3}` at the tail of `I` isn't
  incorrectly assumed to only consume one dimension.
* `Base.to_indices` may fail to infer the returned type. This is the case for `inds2` and
  `inds3` in the first bullet on Julia 1.6.4.
* Specializing by dispatch through method definitions like this:
  `to_indices(::ArrayType, ::Tuple{AxisType,Vararg{Any}}, ::Tuple{::IndexType,Vararg{Any}})`
  require an excessive number of hand written methods to avoid ambiguities. Furthermore, if
  `AxisType` is wrapping another axis that should have unique behavior, then unique parametric
  types need to also be explicitly defined.
* `to_index(axes(A, dim), index)` is called, as opposed to `Base.to_index(A, index)`. The
  `IndexStyle` of the resulting axis is used to allow indirect dispatch on nested axis types
  within `to_index`.
"""
to_indices(A, ::Tuple{}) = ()
@inline function to_indices(a::A, inds::I) where {A,I}
    _to_indices(
        a,
        inds,
        IndexStyle(A),
        static(ndims(A)),
        eachop(_ndims_index, nstatic(Val(known_length(I))), I),
        eachop(_is_splat, nstatic(Val(known_length(I))), I)
    )
end
@generated function _to_indices(A, inds::I, ::S, ::StaticInt{N}, ::NDI, ::IS) where {I,S,N,NDI,IS}
    cnt = zeros(Int, known_length(NDI))
    splat_position = 0
    remaining = N
    for i in 1:known_length(NDI)
        ndi = known(NDI.parameters[i])
        splat = known(IS.parameters[i])
        if splat && splat_position === 0
            splat_position = i
        else
            remaining -= ndi
            cnt[i] = ndi
        end
    end
    if splat_position !== 0
        cnt[splat_position] = max(0, remaining)
    else
        # if there are additional trailing dimensions not consumed by the index then we have
        # to assume it's linear indexing or that these are trailing dimensions.
        cnt[end] += max(0, remaining)
    end

    t = Expr(:tuple)
    dim = 0
    for i in 1:known_length(NDI)
        if i === known_length(NDI) && S <: IndexLinear
            ICall = :LinearIndices
        else
            ICall = :CartesianIndices
        end
        c = cnt[i]
        iexpr = :(@inbounds(getfield(inds, $i))::$(I.parameters[i]))
        if dim === N
            push!(t.args, :(to_index($(ICall)(()), $iexpr)))
        elseif c === 1
            dim += 1
            push!(t.args, :(to_index(@inbounds(getfield(axs, $dim)), $iexpr)))
        else
            subaxs = Expr(:tuple)
            for _ in 1:c
                if dim < N
                    dim += 1
                    push!(subaxs.args, :(@inbounds(getfield(axs, $dim))))
                end
            end
            push!(t.args, :(to_index($(ICall)($subaxs), $iexpr)))
        end
    end
    Expr(:block,
        Expr(:meta, :inline),
        Expr(:(=), :axs, :(lazy_axes(A))),
        :(_flatten_tuples($t))
    )
end
@generated function _flatten_tuples(inds::I) where {I}
    t = Expr(:tuple)
    for i in 1:known_length(I)
        p = I.parameters[i]
        if p <: Tuple
            for j in 1:known_length(p)
                push!(t.args, :(@inbounds(getfield(getfield(inds, $i), $j))))
            end
        else
            push!(t.args, :(@inbounds(getfield(inds, $i))))
        end
    end
    t
end

"""
    to_index([::IndexStyle, ]axis, arg) -> index

Convert the argument `arg` that was originally passed to `getindex` for the dimension
corresponding to `axis` into a form for native indexing (`Int`, Vector{Int}, etc.). New
axis types with unique behavior should use an `IndexStyle` trait:

```julia
to_index(axis::MyAxisType, arg) = to_index(IndexStyle(axis), axis, arg)
to_index(::MyIndexStyle, axis, arg) = ...
```
"""
to_index(x, i::Slice) = i
to_index(x, ::Colon) = indices(x)
# logical indexing
to_index(x, i::AbstractArray{Bool}) = LogicalIndex(i)
to_index(x::LinearIndices, i::AbstractArray{Bool}) = LogicalIndex{Int}(i)
# cartesian indexing
@inline to_index(x, i::CartesianIndices{0}) = i
@inline to_index(x, i::CartesianIndices) = axes(i)
@inline to_index(x, i::CartesianIndex) = Tuple(i)
@inline to_index(x, i::NDIndex) = Tuple(i)
@inline to_index(x, i::AbstractArray{<:AbstractCartesianIndex}) = i
# integer indexing
to_index(x, i::AbstractArray{<:Integer}) = i
to_index(x, @nospecialize(i::StaticInt)) = i
to_index(x, i::Integer) = Int(i)
@inline to_index(x, i) = to_index(IndexStyle(x), x, i)
function to_index(S::IndexStyle, x, i)
    throw(ArgumentError(
        "invalid index: $S does not support indices of type $(typeof(i)) for instances of type $(typeof(x))."
    ))
end

"""
    unsafe_reconstruct(A, data; kwargs...)

Reconstruct `A` given the values in `data`. New methods using `unsafe_reconstruct`
should only dispatch on `A`.
"""
function unsafe_reconstruct(axis::OneTo, data; kwargs...)
    if axis === data
        return axis
    else
        return OneTo(data)
    end
end
function unsafe_reconstruct(axis::UnitRange, data; kwargs...)
    if axis === data
        return axis
    else
        return UnitRange(first(data), last(data))
    end
end
function unsafe_reconstruct(axis::OptionallyStaticUnitRange, data; kwargs...)
    if axis === data
        return axis
    else
        return OptionallyStaticUnitRange(static_first(data), static_last(data))
    end
end
function unsafe_reconstruct(A::AbstractUnitRange, data; kwargs...)
    return static_first(data):static_last(data)
end

"""
    to_axes(A, inds) -> Tuple

Construct new axes given the corresponding `inds` constructed after
`to_indices(A, args) -> inds`. This method iterates through each pair of axes and
indices calling [`to_axis`](@ref).
"""
@inline function to_axes(A, inds::Tuple)
    if ndims(A) === 1
        return (to_axis(axes(A, 1), first(inds)),)
    elseif length(inds) === 1
        return (to_axis(eachindex(IndexLinear(), A), first(inds)),)
    else
        return to_axes(A, axes(A), inds)
    end
end
# drop this dimension
to_axes(A, a::Tuple, i::Tuple{<:Integer,Vararg{Any}}) = to_axes(A, tail(a), tail(i))
to_axes(A, a::Tuple, i::Tuple{I,Vararg{Any}}) where {I} = _to_axes(ndims_index(I), A, a, i)
function _to_axes(::StaticInt{1}, A, axs::Tuple, inds::Tuple)
    return (to_axis(first(axs), first(inds)), to_axes(A, tail(axs), tail(inds))...)
end
@propagate_inbounds function _to_axes(::StaticInt{N}, A, axs::Tuple, inds::Tuple) where {N}
    axes_front, axes_tail = Base.IteratorsMD.split(axs, Val(N))
    if IndexStyle(A) === IndexLinear()
        axis = to_axis(LinearIndices(axes_front), getfield(inds, 1))
    else
        axis = to_axis(CartesianIndices(axes_front), getfield(inds, 1))
    end
    return (axis, to_axes(A, axes_tail, tail(inds))...)
end
to_axes(A, ::Tuple{Ax,Vararg{Any}}, ::Tuple{}) where {Ax} = ()
to_axes(A, ::Tuple{}, ::Tuple{}) = ()

"""
    to_axis(old_axis, index) -> new_axis

Construct an `new_axis` for a newly constructed array that corresponds to the
previously executed `to_index(old_axis, arg) -> index`. `to_axis` assumes that
`index` has already been confirmed to be in bounds. The underlying indices of
`new_axis` begins at one and extends the length of `index` (i.e., one-based indexing).
"""
@inline function to_axis(axis, inds)
    if !can_change_size(axis) &&
       (known_length(inds) !== nothing && known_length(axis) === known_length(inds))
        return axis
    else
        return to_axis(IndexStyle(axis), axis, inds)
    end
end

# don't need to check size b/c slice means it's the entire axis
@inline function to_axis(axis, inds::Slice)
    if can_change_size(axis)
        return copy(axis)
    else
        return axis
    end
end
to_axis(S::IndexLinear, axis, inds) = StaticInt(1):static_length(inds)

"""
    ArrayInterface.getindex(A, args...)

Retrieve the value(s) stored at the given key or index within a collection. Creating
another instance of `ArrayInterface.getindex` should only be done by overloading `A`.
Changing indexing based on a given argument from `args` should be done through,
[`to_index`](@ref), or [`to_axis`](@ref).
"""
function getindex(A, args...)
    inds = to_indices(A, args)
    @boundscheck checkbounds(A, inds...)
    unsafe_getindex(A, inds...)
end
function getindex(A; kwargs...)
    inds = to_indices(A, order_named_inds(dimnames(A), values(kwargs)))
    @boundscheck checkbounds(A, inds...)
    unsafe_getindex(A, inds...)
end
@propagate_inbounds getindex(x::Tuple, i::Int) = getfield(x, i)
@propagate_inbounds getindex(x::Tuple, ::StaticInt{i}) where {i} = getfield(x, i)

## unsafe_getindex ##
function unsafe_getindex(a::A) where {A}
    parent_type(A) <: A && throw(MethodError(unsafe_getindex, (A,)))
    unsafe_getindex(parent(a))
end

# TODO Need to manage index transformations between nested layers of arrays
function unsafe_getindex(a::A, i::CanonicalInt) where {A}
    if IndexStyle(A) === IndexLinear()
        parent_type(A) <: A && throw(MethodError(unsafe_getindex, (A, i)))
        return unsafe_getindex(parent(a), i)
    else
        return unsafe_getindex(a, _to_cartesian(a, i)...)
    end
end
function unsafe_getindex(a::A, i::CanonicalInt, ii::Vararg{CanonicalInt}) where {A}
    if IndexStyle(A) === IndexLinear()
        return unsafe_getindex(a, _to_linear(a, (i, ii...)))
    else
        parent_type(A) <: A && throw(MethodError(unsafe_getindex, (A, i)))
        return unsafe_getindex(parent(a), i, ii...)
    end
end

unsafe_getindex(a, i::Vararg{Any}) = unsafe_get_collection(a, i)

unsafe_getindex(A::Array) = Base.arrayref(false, A, 1)
unsafe_getindex(A::Array, i::CanonicalInt) = Base.arrayref(false, A, Int(i))
@inline function unsafe_getindex(A::Array, i::CanonicalInt, ii::Vararg{CanonicalInt})
    unsafe_getindex(A, _to_linear(A, (i, ii...)))
end

unsafe_getindex(A::LinearIndices, i::CanonicalInt) = Int(i)
unsafe_getindex(A::CartesianIndices, i::CanonicalInt, ii::Vararg{CanonicalInt}) = CartesianIndex(i, ii...)
unsafe_getindex(A::CartesianIndices, i::CanonicalInt) = @inbounds(A[i])

unsafe_getindex(A::ReshapedArray, i::CanonicalInt) = @inbounds(parent(A)[i])
function unsafe_getindex(A::ReshapedArray, i::CanonicalInt, ii::Vararg{CanonicalInt})
    @inbounds(parent(A)[_to_linear(A, (i, ii...))])
end

unsafe_getindex(A::SubArray, i::CanonicalInt) = @inbounds(A[i])
unsafe_getindex(A::SubArray, i::CanonicalInt, ii::Vararg{CanonicalInt}) = @inbounds(A[i, ii...])

# This is based on Base._unsafe_getindex from https://github.com/JuliaLang/julia/blob/c5ede45829bf8eb09f2145bfd6f089459d77b2b1/base/multidimensional.jl#L755.
#=
    unsafe_get_collection(A, inds)

Returns a collection of `A` given `inds`. `inds` is assumed to have been bounds-checked.
=#
function unsafe_get_collection(A, inds)
    axs = to_axes(A, inds)
    dest = similar(A, axs)
    if map(Base.unsafe_length, axes(dest)) == map(Base.unsafe_length, axs)
        Base._unsafe_getindex!(dest, A, inds...)
    else
        Base.throw_checksize_error(dest, axs)
    end
    return dest
end
_ints2range(x::Integer) = x:x
_ints2range(x::AbstractRange) = x
@inline function unsafe_get_collection(A::CartesianIndices{N}, inds) where {N}
    if (length(inds) === 1 && N > 1) || stride_preserving_index(typeof(inds)) === False()
        return Base._getindex(IndexStyle(A), A, inds...)
    else
        return CartesianIndices(to_axes(A, _ints2range.(inds)))
    end
end
@inline function unsafe_get_collection(A::LinearIndices{N}, inds) where {N}
    if length(inds) === 1 && isone(_ndims_index(typeof(inds), static(1)))
        return @inbounds(eachindex(A)[first(inds)])
    elseif stride_preserving_index(typeof(inds)) === True()
        return LinearIndices(to_axes(A, _ints2range.(inds)))
    else
        return Base._getindex(IndexStyle(A), A, inds...)
    end
end

"""
    ArrayInterface.setindex!(A, args...)

Store the given values at the given key or index within a collection.
"""
@propagate_inbounds function setindex!(A, val, args...)
    can_setindex(A) || error("Instance of type $(typeof(A)) are not mutable and cannot change elements after construction.")
    inds = to_indices(A, args)
    @boundscheck checkbounds(A, inds...)
    unsafe_setindex!(A, val, inds...)
end
@propagate_inbounds function setindex!(A, val; kwargs...)
    can_setindex(A) || error("Instance of type $(typeof(A)) are not mutable and cannot change elements after construction.")
    inds = to_indices(A, order_named_inds(dimnames(A), values(kwargs)))
    @boundscheck checkbounds(A, inds...)
    unsafe_setindex!(A, val, inds...)
end

## unsafe_setindex! ##
function unsafe_setindex!(a::A, v) where {A}
    parent_type(A) <: A && throw(MethodError(unsafe_setindex!, (A, v)))
    return unsafe_setindex!(parent(a), v)
end
# TODO Need to manage index transformations between nested layers of arrays
function unsafe_setindex!(a::A, v, i::CanonicalInt) where {A}
    if IndexStyle(A) === IndexLinear()
        parent_type(A) <: A && throw(MethodError(unsafe_setindex!, (A, v, i)))
        return unsafe_setindex!(parent(a), v, i)
    else
        return unsafe_setindex!(a, v, _to_cartesian(a, i)...)
    end
end
function unsafe_setindex!(a::A, v, i::CanonicalInt, ii::Vararg{CanonicalInt}) where {A}
    if IndexStyle(A) === IndexLinear()
        return unsafe_setindex!(a, v, _to_linear(a, (i, ii...)))
    else
        parent_type(A) <: A && throw(MethodError(unsafe_setindex!, (A, v, i, ii...)))
        return unsafe_setindex!(parent(a), v, i, ii...)
    end
end

function unsafe_setindex!(A::Array{T}, v) where {T}
    Base.arrayset(false, A, convert(T, v)::T, 1)
end
function unsafe_setindex!(A::Array{T}, v, i::CanonicalInt) where {T}
    return Base.arrayset(false, A, convert(T, v)::T, Int(i))
end

unsafe_setindex!(a, v, i::Vararg{Any}) = unsafe_set_collection!(a, v, i)

# This is based on Base._unsafe_setindex!.
#=
    unsafe_set_collection!(A, val, inds)

Sets `inds` of `A` to `val`. `inds` is assumed to have been bounds-checked.
=#
unsafe_set_collection!(A, v, i) = Base._unsafe_setindex!(IndexStyle(A), A, v, i...)


import Pkg; Pkg.add(Pkg.PackageSpec(url="https://github.com/JuliaComputing/JuliaAcademyData.jl"))
using JuliaAcademyData; activate("Parallel_Computing")

# # GPUs
#
# The graphics processor in your computer is _itself_ like a mini-computer highly
# tailored for massively and embarassingly parallel operations (like computing how light will bounce
# off of every point on a 3D mesh of triangles).
#
# Of course, recently their utility in other applications has become more clear
# and thus the GPGPU was born.
#
# Just like how we needed to send data to other processes, we need to send our
# data to the GPU to do computations there.

#-

# ## How is a GPU different from a CPU?
#
# This is what a typical consumer CPU looks like:
#
# ![](https://raw.githubusercontent.com/JuliaComputing/JuliaAcademyData.jl/master/courses/Parallel_Computing/images/i7.jpg)
#
# And this is what a GPU looks like:
#
# ![](https://raw.githubusercontent.com/JuliaComputing/JuliaAcademyData.jl/master/courses/Parallel_Computing/images/GK110.jpg)
#
# Each SMX isn't just one "core", each is a _streaming multiprocessor_ capable of running hundreds of threads simultaneously itself.  There are so many threads, in fact, that you reason about them in groups of 32 — called a "warp."  No, no [that warp](https://www.google.com/search?tbm=isch&q=warp&tbs=imgo:1), [this one](https://www.google.com/search?tbm=isch&q=warp%20weaving&tbs=imgo:1).
#
# The card above supports up to 6 warps per multiprocessor, with 32 threads each, times 15 multiprocessors... 2880 threads at a time!
#
# Also note the memory interfaces.
#
# --------------
#
# Each thread is relatively limited — and a warp is almost like a SIMD unit that supports branching. Except it's still only executing one instruction even after a branch:
#
# ![](https://raw.githubusercontent.com/JuliaComputing/JuliaAcademyData.jl/master/courses/Parallel_Computing/images/warp-branch.png)

#-

# You can inspect the installed GPUs with nvidia-smi:

#nb ;nvidia-smi
#jl run(`nvidia-smi`)

# ## Example
#
# The deep learning MNIST example: https://fluxml.ai/experiments/mnist/
#
# This is how it looks on the CPU:

using Flux, Flux.Data.MNIST, Statistics
using Flux: onehotbatch, onecold, crossentropy, throttle
using Base.Iterators: repeated, partition

imgs = MNIST.images()
labels = onehotbatch(MNIST.labels(), 0:9)

## Partition into batches of size 32
train = [(cat(float.(imgs[i])..., dims = 4), labels[:,i])
         for i in partition(1:60_000, 32)]
## Prepare test set (first 1,000 images)
tX = cat(float.(MNIST.images(:test)[1:1000])..., dims = 4)
tY = onehotbatch(MNIST.labels(:test)[1:1000], 0:9)

m = Chain(
        Conv((3, 3), 1=>32, relu),
        Conv((3, 3), 32=>32, relu),
        x -> maxpool(x, (2,2)),
        Conv((3, 3), 32=>16, relu),
        x -> maxpool(x, (2,2)),
        Conv((3, 3), 16=>10, relu),
        x -> reshape(x, :, size(x, 4)),
        Dense(90, 10), softmax)

loss(x, y) = crossentropy(m(x), y)
accuracy(x, y) = mean(onecold(m(x)) .== onecold(y))
## opt = ADAM() # <-- Move Flux.params(m) here!
## @time Flux.train!(loss, Flux.params(m), train[1:10], opt, cb = () -> @show(accuracy(tX, tY)))
opt = ADAM(Flux.params(m), ) # <-- Move Flux.params(m) here!
Flux.train!(loss, train[1:1], opt, cb = () -> @show(accuracy(tX, tY)))
@time Flux.train!(loss, train[1:10], opt, cb = () -> @show(accuracy(tX, tY)))

# Now let's re-do it on a GPU. "All" it takes is moving the data there with `gpu`!

include(datapath("scripts/fixupCUDNN.jl")) # JuliaBox uses an old version of CuArrays; this backports a fix for it
gputrain = gpu.(train[1:10])
gpum = gpu(m)
gputX = gpu(tX)
gputY = gpu(tY)
gpuloss(x, y) = crossentropy(gpum(x), y)
gpuaccuracy(x, y) = mean(onecold(gpum(x)) .== onecold(y))
gpuopt = ADAM(Flux.params(gpum), )
Flux.train!(gpuloss, gpu.(train[1:1]), gpuopt, cb = () -> @show(gpuaccuracy(gputX, gputY)))
@time Flux.train!(gpuloss, gputrain, gpuopt, cb = () -> @show(gpuaccuracy(gputX, gputY)))

# ## Defining your own GPU kernels
#
# So that's leveraging Flux's ability to work with GPU arrays — which is magical
# and awesome — but you don't always have a library to lean on like that.
# How might you define your own GPU kernel?
#
# Recall the monte carlo pi example:

function serialpi(n)
    inside = 0
    for i in 1:n
        x, y = rand(), rand()
        inside += (x^2 + y^2 <= 1)
    end
    return 4 * inside / n
end

# How could we express this on the GPU?

using CuArrays.CURAND
function findpi_gpu(n)
    4 * sum(curand(Float64, n).^2 .+ curand(Float64, n).^2 .<= 1) / n
end
findpi_gpu(10_000_000)

#-

using BenchmarkTools
@btime findpi_gpu(10_000_000)
@btime serialpi(10_000_000)

# That leans on broadcast to build the GPU kernel — and is creating three arrays
# in the process — but it's still much faster than our serial pi from before.

#-

# In general, using CuArrays and broadcast is one of the best ways to just
# get everything to work. If you really want to get your hands dirty, you
# can use [CUDAnative.jl](https://github.com/JuliaGPU/CUDAnative.jl) to manually specify exactly how everything works,
# but be forewarned, it's not for the [faint at heart](https://github.com/JuliaGPU/CUDAnative.jl/blob/master/examples/reduce/reduce.jl)! (If you've done CUDA
# programming in C or C++, it's very similar.)


# ------------------------------------------------------------------------------------------
# # Julia for Data Science - Data
# Prepared by [@nassarhuda](https://github.com/nassarhuda)! 😃
#
# In the next few notebooks, we will discuss why *Julia* is the tool you want to use for
# your data science applications.
#
# We will cover the following:
# 1. Reading and writing files
# 1. DataFrames
# 1. RDatasets
# 1. FileIO
# 1. File types
#
#
# ### Data: Build a strong relationship with your data.
# Every data science task has one main ingredient, the _data_! Most likely, you want to use
# your data to learn something new. But before the _new_ part, what about the data you
# already have? Let's make sure you can **read** it, **store** it, and **understand** it
# before you start using it.
#
# Julia makes this step really easy with data structures and packages to process the data,
# as well as existing functions that are readily usable on your data.
#
# The goal of this first part is get you acquainted with some Julia's tools to manage your
# data.
# ------------------------------------------------------------------------------------------

# ------------------------------------------------------------------------------------------
# **Reading and writing to files is really easy in Julia.**

#
# To see this, let's download a csv file from github that we can work with.
#
# Note: `download` depends on external tools such as curl, wget or fetch. So you must have
# one of these.
# ------------------------------------------------------------------------------------------

P = download("https://raw.githubusercontent.com/nassarhuda/easy_data/master/programming_languages.csv","programminglanguages.csv")

# ------------------------------------------------------------------------------------------
# We can use shell commands like `ls` in Julia by preceding them with a semicolon.
# ------------------------------------------------------------------------------------------

;ls

# ------------------------------------------------------------------------------------------
# And there's the *.csv file we downloaded!
#
# By default, `readdlm` will fill an array with the data stored in the input .csv file. If
# we set the keyword argument `header` to `true`, we'll get a second output array for the
# headers.
# ------------------------------------------------------------------------------------------

using DelimitedFiles
P,H = readdlm("programminglanguages.csv",header=true)

P

# ------------------------------------------------------------------------------------------
# You can use different delimiters with the function `readdlm`.
#
# To write to files, we can use `writedlm`.

#
# Let's write this same data to a file with a different delimiter.
# ------------------------------------------------------------------------------------------

writedlm("programming_languages_data.txt", P, '-')

# ------------------------------------------------------------------------------------------
# We can now check that this worked using a shell command to glance at the file,
# ------------------------------------------------------------------------------------------

;head -10 programming_languages_data.txt

# ------------------------------------------------------------------------------------------
# and also check that we can use `readdlm` to read our new text file correctly.
# ------------------------------------------------------------------------------------------

P_new_delim = readdlm("programming_languages_data.txt", '-');
P == P_new_delim

# ------------------------------------------------------------------------------------------
# ### DataFrames!
# *Shout out to R fans!*
# One other way to play around with data in Julia is to use a DataFrame.
#
# This requires loading the `DataFrames` package.
#
# Run this command to install all the packages used in the "Julia for Data Science" project
# -- (those packages are listed in this file: [`Project.toml`](/edit/introductory-
# tutorials/broader-topics-and-ecosystem/intro-to-julia-for-data-science/Project.toml)):
# ------------------------------------------------------------------------------------------

] instantiate

using DataFrames
df = DataFrame(year = P[:,1], language = P[:,2])

# ------------------------------------------------------------------------------------------
# You can access columns by header name, or column index.
#
# In this case, `df[1]` is equivalent to `df.year` or `df[!, :year]`.
#
# Note that if we want to index columns by header name, we precede the header name with a
# colon. In Julia, this means that the header names are treated as *symbols*.
# ------------------------------------------------------------------------------------------

df.year

# ------------------------------------------------------------------------------------------
# **`DataFrames` provides some handy features when dealing with data**
#
# First, it uses julia's "missing" type.
# ------------------------------------------------------------------------------------------

a = missing
typeof(a)

# ------------------------------------------------------------------------------------------
# Let's see what happens when we try to add a "missing" type to a number
# ------------------------------------------------------------------------------------------

a + 1

# ------------------------------------------------------------------------------------------
# ### RDatasets
#
# We can use RDatasets to play around with pre-existing datasets
# ------------------------------------------------------------------------------------------

using RDatasets
iris = dataset("datasets", "iris")

# ------------------------------------------------------------------------------------------
# Note that data loaded with `dataset` is stored as a DataFrame. 😃
# ------------------------------------------------------------------------------------------

typeof(iris)

# ------------------------------------------------------------------------------------------
# `DataFrames` provides the `describe` can give you quick statistics about each column in
# your dataframe
# ------------------------------------------------------------------------------------------

describe(iris)

# ------------------------------------------------------------------------------------------
# You can create your own dataframe quickly as follows
# ------------------------------------------------------------------------------------------

foods = ["apple", "cucumber", "tomato", "banana"]
calories = [missing,47,22,105]
typeof(calories)

using Statistics
mean(calories)

# ------------------------------------------------------------------------------------------
# `missing` ruins everything! 😑
# ------------------------------------------------------------------------------------------

mean(skipmissing(calories))

# ------------------------------------------------------------------------------------------
# In fact, `describe' will drop these values too
# ------------------------------------------------------------------------------------------

describe(DataFrame(c=calories))

# ------------------------------------------------------------------------------------------
# Note that `typeof(calories)` is `Vector{Union{Missing, Int64},1}`
#
# If we want to replace all `missing` values a default value, we can do it like this:
# ------------------------------------------------------------------------------------------

newcalories = replace(calories, missing => 0)

# ------------------------------------------------------------------------------------------
# Now let's create a `DataFrame` that shows foods and their calories from two `DataArray`s!
# ------------------------------------------------------------------------------------------

dataframe_calories = DataFrame(item=foods,calories=calories)

# ------------------------------------------------------------------------------------------
# Let's generate a second `DataFrame` that shows foods and their prices.
# ------------------------------------------------------------------------------------------

prices = [0.85,1.6,0.8,0.6,]

dataframe_prices = DataFrame(item=foods,price=prices)

# ------------------------------------------------------------------------------------------
# We can also `join` these two dataframes together because they share a common column,
# `item`.
# ------------------------------------------------------------------------------------------

DF = join(dataframe_calories,dataframe_prices,on=:item)

# ------------------------------------------------------------------------------------------
# Note that we used the keyword argument `on` to say that we wanted to join these dataframes
# together based on the `item` column.
# ------------------------------------------------------------------------------------------

# ------------------------------------------------------------------------------------------
# ### FileIO
# ------------------------------------------------------------------------------------------

using FileIO
julialogo = download("https://avatars0.githubusercontent.com/u/743164?s=200&v=4","julialogo.png")

# ------------------------------------------------------------------------------------------
# Again, let's check that this download worked!
# ------------------------------------------------------------------------------------------

;ls

# ------------------------------------------------------------------------------------------
# Next, let's load the Julia logo, stored as a .png file
# ------------------------------------------------------------------------------------------

X1 = load("julialogo.png")

# ------------------------------------------------------------------------------------------
# We see below that Julia stores this logo as an array of colors.
# ------------------------------------------------------------------------------------------

@show typeof(X1);
@show size(X1);

# ------------------------------------------------------------------------------------------
# ### File types
# In Julia, many file types are supported so you do not have to transfer a file you got from
# another language to a text file before you read it.
#
# *Some packages that achieve this:*
# MAT CSV NPZ JLD FASTAIO
#
#
# Let's try using MAT to write a file that stores a matrix.
# ------------------------------------------------------------------------------------------

using MAT

A = rand(5,5)
matfile = matopen("densematrix.mat", "w")
write(matfile, "A", A)
close(matfile)

# ------------------------------------------------------------------------------------------
# Now try opening densematrix.mat with MATLAB!
# ------------------------------------------------------------------------------------------

newfile = matopen("densematrix.mat")
read(newfile,"A")

names(newfile)

close(newfile)


# Copyright (c) 2019 Arpit Bhatia and contributors                               #src
#                                                                                #src
# Permission is hereby granted, free of charge, to any person obtaining a copy   #src
# of this software and associated documentation files (the "Software"), to deal  #src
# in the Software without restriction, including without limitation the rights   #src
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell      #src
# copies of the Software, and to permit persons to whom the Software is          #src
# furnished to do so, subject to the following conditions:                       #src
#                                                                                #src
# The above copyright notice and this permission notice shall be included in all #src
# copies or substantial portions of the Software.                                #src
#                                                                                #src
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR     #src
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,       #src
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE    #src
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER         #src
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,  #src
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE  #src
# SOFTWARE.                                                                      #src

# # Getting started with JuMP

# This tutorial is aimed at providing a quick introduction to writing JuMP code.
# If you're new to Julia, you should start with [Getting started with Julia](@ref).

# ## What is JuMP?

# JuMP ("Julia for Mathematical Programming") is an open-source modeling
# language that is embedded in Julia. It allows users to users formulate various
# classes of optimization problems (linear, mixed-integer, quadratic, conic
# quadratic, semidefinite, and nonlinear) with easy-to-read code. These problems
# can then be solved using state-of-the-art open-source and commercial solvers.

# JuMP also makes advanced optimization techniques easily accessible from a
# high-level language.

# ## Installation

# JuMP is a package for Julia. From Julia, JuMP is installed by using the
# built-in package manager.

# ```julia
# import Pkg
# Pkg.add("JuMP")
# ```

# You also need to include a Julia package which provides an appropriate solver.
# One such solver is `GLPK.Optimizer`, which is provided by the
# [GLPK.jl package](https://github.com/JuliaOpt/GLPK.jl).
# ```julia
# import Pkg
# Pkg.add("GLPK")
# ```
# See [Installation Guide](@ref) for a list of other solvers you can use.

# ## An example

# Let's try to solve the following linear programming problem by using JuMP and
# GLPK. We will first look at the complete code to solve the problem and then go
# through it step by step.

# ```math
# \begin{aligned}
# & \min & 12x + 20y \\
# & \;\;\text{s.t.} & 6x + 8y \geq 100 \\
# & & 7x + 12y \geq 120 \\
# & & x \geq 0 \\
# & & y \in [0, 3] \\
# \end{aligned}
# ```

using JuMP
using GLPK
model = Model(GLPK.Optimizer)
@variable(model, x >= 0)
@variable(model, 0 <= y <= 3)
@objective(model, Min, 12x + 20y)
@constraint(model, c1, 6x + 8y >= 100)
@constraint(model, c2, 7x + 12y >= 120)
print(model)
optimize!(model)
@show termination_status(model)
@show primal_status(model)
@show dual_status(model)
@show objective_value(model)
@show value(x)
@show value(y)
@show shadow_price(c1)
@show shadow_price(c2)
nothing #hide

# ## Step-by-step

# Once JuMP is installed, to use JuMP in your programs, we just need to write:

using JuMP

# We also need to include a Julia package which provides an appropriate solver.
# We want to use `GLPK.Optimizer` here which is provided by the `GLPK.jl`
# package.

using GLPK

# A model object is a container for variables, constraints, solver options, etc.
# Models are created with the [`Model`](@ref) function. The model can be created
# with an optimizer attached with default arguments by calling the constructor
# with the optimizer type, as follows:

model = Model(GLPK.Optimizer)

# Variables are modelled using [`@variable`](@ref):

@variable(model, x >= 0)

# They can have lower and upper bounds.

@variable(model, 0 <= y <= 30)

# The objective is set using [`@objective`](@ref):

@objective(model, Min, 12x + 20y)

# Constraints are modelled using [`@constraint`](@ref). Here `c1` and `c2` are
# the names of our constraint.

@constraint(model, c1, 6x + 8y >= 100)

#-

@constraint(model, c2, 7x + 12y >= 120)

#- Call `print` to display the model:

print(model)

# To solve the optimization problem, call the [`optimize!`] function.

optimize!(model)

# !!! info
#     The `!` after optimize is just part of the name. It's nothing special.
#     Julia has a convention that functions which mutate their arguments should
#     end in `!`. A common example is `push!`.

# Now let's see what information we can query about the solution.

# [`termination_status`](@ref) tells us why the solver stopped:

termination_status(model)

# In this case, the solver found an optimal solution. We should also check
# [`primal_status`](@ref) to see if the solver found a primal feasible point:

primal_status(model)

# and [`dual_status`](@ref) to see if the solver found a dual feasible point:

dual_status(model)

# Now we know that our solver found an optimal solution, and has a primal and a
# dual solution to query.

# Query the objective value using [`objective_value`](@ref):

objective_value(model)

# The primal solution using [`value`](@ref):

value(x)

#-

value(y)

# and the dual solution using [`shadow_price`](@ref):

shadow_price(c1)

#-

shadow_price(c2)