import Pkg; Pkg.add(Pkg.PackageSpec(url="https://github.com/JuliaComputing/JuliaAcademyData.jl"))
using JuliaAcademyData; activate("Foundations of machine learning")

## Run this cell to load the graphics packages
using Plots; gr()

#-

## Run this cell to use a couple of definitions from the previous Tools notebook

σ(x) = 1 / (1 + exp(-x))
f(x, w) = σ(w * x)

# ## Multiple function parameters
#
# In notebook 6, we saw how we could adjust a parameter to make a curve fit a single data point. What if there is more data to fit?
#
# We'll see that as we acquire more data to fit, it can sometimes be useful to add complexity to our model via additional parameters.

#-

# ## Adding more data

#-

# Suppose there are now two data points to fit, the previous $(x_0, y_0)$ used in notebook 6 and also $(x_1, y_1) = (-3, 0.3)$.

#-

# #### Exercise 1
#
# Make an interactive plot of the function $f_w$ together with the two data points. Can you make the graph of $f_w$ pass through *both* data points at the *same* time?

x0, y0 = 2, 0.8
x1, y1 = -3, 0.3

w = 5.0 # try manipulating w between 0 and 10
plot(x->f(x, w), -5, 5, ylim=(0,1))
scatter!([x0, x1], [y0, y1])

# You should have found that it's actually *impossible* to fit both data points at the same time! The best we could do is to *minimise* how far away the function is from the data. To do so, we need to somehow balance the distance from each of the two data points.

#-

# #### Exercise 2
#
# Play with the slider to find the value $w^*$ of $w$ that you think has the "least error" in an intuitive sense.

w=0.45
plot(x->f(x, w), -5, 5, ylim=(0,1))
scatter!([x0, x1], [y0, y1])

# ## Defining a loss function

#-

# How can we *quantify* some kind of overall measure of the distance from *all* of the data? Well, we just need to define a new loss function! One way to do so would be to sum up the loss functions for each data point, i.e. the sum of the squared vertical distances from the graph to each data point:
#
# $$L(w) = [y_0 - f_w(x_0)]^2 + [y_1 - f_w(x_1)]^2.$$
#
# Since the two pieces that we are adding have the same mathematical "shape" or structure, we can abbreviate this by writing it as a sum:
#
# $$L(w) = \sum_{i=0}^1 [y_i - f_w(x_i)]^2.$$

#-

# So now we want to find the value $w^*$ of $w$ that minimizes this new function $L$!

#-

# #### Exercise 3
#
# Make a visualization to show the function $f_w$ and to visualize the distance from each of the data points.

#-

# #### Solution

xs = [2, -3]
ys = [0.8, 0.3]

L(w) = sum( (ys .- f.(xs, w)) .^ 2 )

w = 0.0 # Try manipulating w between -2 and 2

    plot(x->f(x, w), -5, 5, ylims=(0, 1), lw = 3, label="f_w")

    scatter!(xs, ys, label="data")

    for i in 1:2
        plot!([xs[i], xs[i]], [ys[i], f(xs[i], w)], c=:green)
    end

    title!("L(w) =  $(round(L(w); sigdigits = 5))")

end

# #### Exercise 4
#
# After playing with this for a while, it is intuitively obvious that we cannot make the function pass through both data points for any value of $w$. In other words, our loss function, `L(w)`, is never zero.
#
# What is the minimum value of `L(w)` that you can find by altering `w`? What is the corresponding value of `w`?

w = 0.42
plot(x->f(x, w), -5, 5, ylims=(0, 1), lw = 3, label="f_w")

scatter!(xs, ys, label="data")

for i in 1:2
    plot!([xs[i], xs[i]], [ys[i], f(xs[i], w)], c=:green)
end

title!("L(w) =  $(round(L(w); sigdigits = 5))")

# ### Sums in Julia

#-

# To generate the above plot we used the `sum` function. `sum` can add together all the elements of a collection or range, or it can add together the outputs of applying a function to all the elements of a collection or range.
#
# Look up the docs for `sum` via
#
# ```julia
# ?sum
# ```
# if you need more information.

#-

# #### Exercise 5
#
# Use `sum` to add together all integers in the range 1 to 16, inclusive. What is the result?

#-

# #### Solution

sum(1:16)

# #### Exercise 6
#
# What is the sum of the absolute values of all integers between -3 and 3? Use `sum` and the `abs` function.

#-

# #### Solution

sum(abs, -3:3)

# ## What does the loss function $L$ look like?

#-

# In our last attempt to minimize `L(w)` by varying `w`, we saw that `L(w)` always seemed to be greater than 0.  Is that true? Let's plot it to find out!

#-

# #### Exercise 7
#
# Plot the new loss function $L(w)$ as a function of $w$.

#-

# #### Solution

plot(L, -2, 2, xlabel="w", ylabel="L(w)", ylims=(0, 1.2))

# ### Features of the loss function

#-

# The first thing to notice is that $L$ is always positive. Since it is the sum of squares, and squares cannot be negative, the sum cannot be negative either!
#
# However, we also see that although $L$ dips close to $0$ for a single, special value $w^* \simeq 0.4$, it never actually *reaches* 0! Again we could zoom in on that region of the graph to estimate it more precisely.

#-

# We might start suspecting that there should be a better way of using the computer to minimize $L$ to find the location $w^*$ of the minimum, rather than still doing everything by eye. Indeed there is, as we will see in the next two notebooks!

#-

# ## Adding more parameters to the model

#-

# If we add more parameters to a function, we may be able to improve how it fits to data. For example, we could define a new function $g$ with another parameter, a shift or **bias**:
#
# $$g(x; w, b) := \sigma(w \, x + b).$$

g(x, w, b) = σ(w*x + b)

# #### Exercise 8
#
# Make an interactive visualization that allows you to change $w$ and $b$. Play with the sliders to try and fit *both* data points at once!

#-

# #### Solution

xs = [2, -3]
ys = [0.8, 0.3]

L2D(w, b) = sum( (ys .- g.(xs, w, b)) .^ 2)

w = 0.0 # Try manipulating w between -2 and 2
b = 0.0 # Try manipulating b between -2 and 2
plot(x -> g(x, w, b), -5, 5, ylims=(0, 1), lw=3)

scatter!(xs, ys)

for i in 1:2
    plot!([xs[i], xs[i]], [ys[i], g(xs[i], w, b)])
end

title!("L2D(w, b) = $(L2D(w, b))")

# You should be able to convince yourself that we can now make the curve pass through both points simultaneously.

#-

# #### Exercise 9
#
# For what values of `w` and `b` does the line pass through both points?

w = 0.45
b = 0.48
plot(x -> g(x, w, b), -5, 5, ylims=(0, 1), lw=3)

scatter!(xs, ys)

for i in 1:2
    plot!([xs[i], xs[i]], [ys[i], g(xs[i], w, b)])
end

title!("L2D(w, b) = $(L2D(w, b))")

# ## Fitting both data points: a loss function

#-

# Following the procedure that we used when we had a single parameter, we can think of the fitting procedure once again as minimizing a suitable *loss function*. The expression for the loss function is almost the same, except that now $f$ has two parameters $w$ and $b$, so the loss function $L_2$ is itself a function of $w$ *and* $b$:

#-

# $$L_2(w, b) = \sum_i [y_i - f(x_i; w, b)]^2.$$

#-

# So we want to minimize this loss function over *both* of the parameters $w$ and $b$! Let's plot it.

#-

# #### Exercise 10
#
# Define the function `L2` in Julia.

#-

# #### Solution

L2(w, b) = sum( (ys .- g.(xs, w, b)) .^ 2 )

# To plot the loss as a function of *two* variables (the weights *and* the bias), we will make use of the `surface` function.  To get a nice interactive 3D plot, we will use the PlotlyJS "backend" (plotting engine):

gr()  # load the backend

#-

ws = -2:0.05:2
bs = -2:0.05:2

contour(ws, bs, L2, levels=0.001:0.025:1.25, xlabel="value of w", ylabel="value of b")

# We can see that indeed there is a unique point $(w^*, b^*)$ where the function $L_2$ attains its minimum. You can try different ranges for the $x$, $y$ and $z$ coordinates to get a better view.

#-

# ## More data

#-

# If we add more data, however, we will again not be able to fit all of the data; we will only be able to attain a "best fit".
#
# Let's create `xs` and `ys` with some more data:

xs = [2, -3, -1, 1]
ys = [0.8, 0.3, 0.4, 0.4]

# #### Exercise 11
#
# a) Make an interactive visualization of the function $f$ and the data. Try to find the values of $w$ and $b$ that give the best fit.
#
# b) Define the loss function and plot it.

#-

# #### Solution

gr()

#-

L2(w, b) = sum( (ys .- g.(xs, w, b)) .^ 2)

w = 0.0 # Try manipulating w between -2 and 2
b = 0.0 # Try manipulating b between -2 and 2
plot(x->g(x, w, b), -5, 5, ylims=(-0.2, 1))

scatter!(xs, ys)

for i in 1:length(xs)
    plot!([xs[i], xs[i]], [ys[i], g(xs[i], w, b)])
end

title!("L2(w, b) = $(round(L2(w, b); sigdigits = 5))")

# Let's define the loss function that we're using above so that we can plot it as a function of the parameters `w` and `b`.

#-

# #### Exercise 12
#
# We've seen the loss function, $$L_{2D}(w, b) = \sum_i[\mathrm{ys}_i - g(\mathrm{xs}_i, w, b)]^2,$$ written in Julia as
#
# ```julia
# L2(w, b) = sum( (ys .- g.(xs, w, b)) .^ 2 )
# ```
#
# a few times now. To ensure you understand what this function is doing, implement your own loss function using the commented code below. Do this without using `sum`, `abs2`, or broadcasting dot syntax (for example, `.-`). Hint: you'll want to use a `for` loop to do this.

function myL2(w, b)
    loss = 0.0
    for (x, y) in zip(xs, ys)
        loss += (y - g(x, w, b))^2
    end
    return loss
end

# Now that you've defined `L2`, we can plot it using either `surface` or `contour` to see how the loss changes as `w` and `b` change!

## plotlyjs() # Use Plotly for 3d surface plots (if possible)

ws = -2:0.05:2
bs = -2:0.05:2

## surface(ws, bs, L2, alpha=0.8, zlims=(0,3))  # alpha gives transparency
contour(ws, bs, L2, levels=0.055:0.05:1.5, xlabel="value of w", ylabel="value of b")

Courses/Foundations of machine learning/0700.Model-complexity-quiz.jl


#Partitions

export
    integer_partitions,
    ncpartitions,
    partitions,
    prevprod
    #nextprod,

#integer partitions

struct IntegerPartitions
    n::Int
end

Base.length(p::IntegerPartitions) = npartitions(p.n)

function Base.iterate(p::IntegerPartitions, xs = Int[])
    length(xs) == p.n && return
    xs = nextpartition(p.n,xs)
    (xs, xs)
end

"""
    partitions(n)

Generate all integer arrays that sum to `n`. Because the number of partitions can be very
large, this function returns an iterator object. Use `collect(partitions(n))` to get an
array of all partitions. The number of partitions to generate can be efficiently computed
using `length(partitions(n))`.
"""
partitions(n::Integer) = IntegerPartitions(n)

function nextpartition(n, as)
    isempty(as) && return Int[n]

    xs = similar(as, 0)
    sizehint!(xs, length(as) + 1)

    for i = 1:length(as)-1
        if as[i+1] == 1
            x = as[i]-1
            push!(xs, x)
            n -= x
            while n > x
                push!(xs, x)
                n -= x
            end
            push!(xs, n)

            return xs
        end
        push!(xs, as[i])
        n -= as[i]
    end
    push!(xs, as[end]-1)
    push!(xs, 1)

    xs
end

let _npartitions = Dict{Int,Int}()
    global npartitions
    function npartitions(n::Int)
        if n < 0
            0
        elseif n < 2
            1
        elseif (np = get(_npartitions, n, 0)) > 0
            np
        else
            np = 0
            sgn = 1
            for k = 1:n
                np += sgn * (npartitions(n - (k*(3k-1)) >> 1) + npartitions(n - (k*(3k+1)) >> 1))
                sgn = -sgn
            end
            _npartitions[n] = np
        end
    end
end

# Algorithm H from TAoCP 7.2.1.4
# Partition n into m parts
# in colex order (lexicographic by reflected sequence)

struct FixedPartitions
    n::Int
    m::Int
end

Base.length(f::FixedPartitions) = npartitions(f.n,f.m)

"""
    partitions(n, m)

Generate all arrays of `m` integers that sum to `n`. Because the number of partitions can
be very large, this function returns an iterator object. Use `collect(partitions(n, m))` to
get an array of all partitions. The number of partitions to generate can be efficiently
computed using `length(partitions(n, m))`.
"""
partitions(n::Integer, m::Integer) =
    n >= 1 && m >= 1 ?
        FixedPartitions(n, m) :
        throw(DomainError((n, m), "n and m must be positive"))

function Base.iterate(f::FixedPartitions, s::Vector{Int} = Int[])
    f.m <= f.n || return
    if !isempty(s)
        (f.m == 1 || s[1]-1 <= s[end]) && return
    end

    xs = nextfixedpartition(f.n,f.m,s)
    (xs, xs)
end

function nextfixedpartition(n, m, bs)
    as = copy(bs)
    if isempty(as)
        # First iteration
        as = [n-m+1; ones(Int, m-1)]
    elseif as[2] < as[1]-1
        # Most common iteration
        as[1] -= 1
        as[2] += 1
    else
        # Iterate
        j = 0
        s = as[1]+as[2]-1
        for jj = 3:m # TODO: use `for outer j = ...` on 0.7
            j = jj
            as[jj] < as[1]-1 && break
            s += as[jj]
        end
        x = as[j] += 1
        for k = j-1:-1:2
            as[k] = x
            s -= x
        end
        as[1] = s
    end

    return as
end

let _nipartitions = Dict{Tuple{Int,Int},Int}()
    global npartitions
    function npartitions(n::Int, m::Int)
        if n < m || m == 0
            0
        elseif n == m
            1
        elseif (np = get(_nipartitions, (n,m), 0)) > 0
            np
        else
            _nipartitions[(n, m)] = npartitions(n-1, m-1) + npartitions(n-m, m)
        end
    end
end

# Algorithm H from TAoCP 7.2.1.5
# Set partitions

struct SetPartitions{T<:AbstractVector}
    s::T
end

Base.length(p::SetPartitions) = nsetpartitions(length(p.s))

"""
    partitions(s::AbstractVector)

Generate all set partitions of the elements of an array `s`, represented as arrays of
arrays. Because the number of partitions can be very large, this function returns an
iterator object. Use `collect(partitions(s))` to get an array of all partitions. The
number of partitions to generate can be efficiently computed using
`length(partitions(s))`.
"""
partitions(s::AbstractVector) = SetPartitions(s)

function Base.iterate(p::SetPartitions)
    n = length(p.s)
    iterate(p, (zeros(Int32, n), ones(Int32, n-1), n, 1))
end
function Base.iterate(p::SetPartitions, s)
    s[1][1] > 0 && return
    nextsetpartition(p.s, s...)
end

function nextsetpartition(s::AbstractVector, a, b, n, m)
    function makeparts(s, a, m)
        temp = [similar(s, 0) for k = 0:m]
        for i = 1:n
            push!(temp[a[i]+1], s[i])
        end
        filter!(!isempty, temp)
    end

    isempty(s) && return ([s], ([1], Int[], n, 1))

    part = makeparts(s,a,m)

    if a[end] != m
        a[end] += 1
    else
        j = 0
        for jj = n-1:-1:1
            j = jj
            a[jj] == b[jj] || break
        end
        a[j] += 1
        m = b[j] + (a[j] == b[j])
        for k = j+1:n-1
            a[k] = 0
            b[k] = m
        end
        a[end] = 0
    end

    return (part, (a,b,n,m))
end

let _nsetpartitions = Dict{Int,Int}()
    global nsetpartitions
    function nsetpartitions(n::Int)
        if n < 0
            0
        elseif n < 2
            1
        elseif (wn = get(_nsetpartitions, n, 0)) > 0
            wn
        else
            wn = 0
            for k = 0:n-1
                wn += binomial(n-1,k)*nsetpartitions(n-1-k)
            end
            _nsetpartitions[n] = wn
        end
    end
end

struct FixedSetPartitions{T<:AbstractVector}
    s::T
    m::Int
end

Base.length(p::FixedSetPartitions) = nfixedsetpartitions(length(p.s),p.m)

"""
    partitions(s::AbstractVector, m::Int)

Generate all set partitions of the elements of an array `s` into exactly `m` subsets,
represented as arrays of arrays. Because the number of partitions can be very large,
this function returns an iterator object. Use `collect(partitions(s, m))` to get
an array of all partitions. The number of partitions into `m` subsets is equal to the
Stirling number of the second kind, and can be efficiently computed using
`length(partitions(s, m))`.
"""
partitions(s::AbstractVector, m::Int) =
    length(s) >= 1 && m >= 1 ?
        FixedSetPartitions(s, m) :
        throw(DomainError((length(s), m), "length(s) and m must be positive"))

function Base.iterate(p::FixedSetPartitions)
    n = length(p.s)
    m = p.m
    state = m <= n ? (vcat(ones(Int, n-m),1:m), vcat(1,n-m+2:n), n) : (Int[], Int[], n)
    # state consists of:
    # vector a of length n describing to which partition every element of s belongs
    # vector b of length n describing the first index b[i] that belongs to partition i
    # integer n

    iterate(p, state)
end

function Base.iterate(p::FixedSetPartitions, s)
    (isempty(s[1]) || s[1][1] > 1) && return
    nextfixedsetpartition(p.s,p.m, s...)
end

function nextfixedsetpartition(s::AbstractVector, m, a, b, n)
    function makeparts(s, a)
        part = [similar(s, 0) for k = 1:m]
        for i = 1:n
            push!(part[a[i]], s[i])
        end
        return part
    end

    part = makeparts(s,a)

    if m == 1
        a[1] = 2
        return (part, (a, b, n))
    end

    if a[end] != m
        a[end] += 1
    else
        j = k = 0
        for jj = n-1:-1:1
            j = jj
            if a[j] < m && b[a[j]+1] < j
                break
            end
        end
        if j > 1
            a[j] += 1
            for p = j+1:n
                if b[a[p]] != p
                    a[p] = 1
                end
            end
        else
            for kk = m:-1:2
                k = kk
                if b[k-1] < b[k] - 1
                    break
                end
            end
            b[k] -= 1
            b[k+1:m] = n-m+k+1:n
            a[1:n] .= 1
            a[b] = 1:m
        end
    end

    return (part, (a,b,n))
end

function nfixedsetpartitions(n::Int, m::Int)
    numpart = 0
    for k = 0:m
        numpart += (-1)^(m-k) * binomial(m, k) * (k^n)
    end
    numpart = div(numpart, factorial(m))
    return numpart
end

# TODO: Base.DSP is no longer a thing in Julia 0.7
#This function is still defined in Base because it is being used by Base.DSP
#"""
#Next integer not less than `n` that can be written as $\prod k_i^{p_i}$ for integers $p_1$, $p_2$, etc.
#
#For a list of integers i1, i2, i3, find the smallest
#    i1^n1 * i2^n2 * i3^n3 >= x
#for integer n1, n2, n3
#"""
#function nextprod(a::Vector{Int}, x)
#    if x > typemax(Int)
#        throw(ArgumentError("unsafe for x > typemax(Int), got $x"))
#    end
#    k = length(a)
#    v = ones(Int, k)                  # current value of each counter
#    mx = [nextpow(ai,x) for ai in a]  # maximum value of each counter
#    v[1] = mx[1]                      # start at first case that is >= x
#    p::widen(Int) = mx[1]             # initial value of product in this case
#    best = p
#    icarry = 1
#
#    while v[end] < mx[end]
#        if p >= x
#            best = p < best ? p : best  # keep the best found yet
#            carrytest = true
#            while carrytest
#                p = div(p, v[icarry])
#                v[icarry] = 1
#                icarry += 1
#                p *= a[icarry]
#                v[icarry] *= a[icarry]
#                carrytest = v[icarry] > mx[icarry] && icarry < k
#            end
#            if p < x
#                icarry = 1
#            end
#        else
#            while p < x
#                p *= a[1]
#                v[1] *= a[1]
#            end
#        end
#    end
#    best = mx[end] < best ? mx[end] : best
#    return Int(best)  # could overflow, but best to have predictable return type
#end

"""
    prevprod(a::Vector{Int}, x)

Previous integer not greater than `x` that can be written as ``\\prod k_i^{p_i}`` for
integers ``p_1``, ``p_2``, etc.

For integers ``i_1``, ``i_2``, ``i_3``, this is equivalent to finding the largest ``x``
such that
```math
i_1^n_1 i_2^n_2 i_3^n_3 \\leq x
```
for integers ``n_1``, ``n_2``, ``n_3``.
"""
function prevprod(a::Vector{Int}, x)
    if x > typemax(Int)
        throw(ArgumentError("unsafe for x > typemax(Int), got $x"))
    end
    k = length(a)
    v = ones(Int, k)                  # current value of each counter
    mx = [nextpow(ai,x) for ai in a]  # allow each counter to exceed p (sentinel)
    first = Int(prevpow(a[1], x))     # start at best case in first factor
    v[1] = first
    p::widen(Int) = first
    best = p
    icarry = 1

    while v[end] < mx[end]
        while p <= x
            best = p > best ? p : best
            p *= a[1]
            v[1] *= a[1]
        end
        if p > x
            carrytest = true
            while carrytest
                p = div(p, v[icarry])
                v[icarry] = 1
                icarry += 1
                p *= a[icarry]
                v[icarry] *= a[icarry]
                carrytest = v[icarry] > mx[icarry] && icarry < k
            end
            if p <= x
                icarry = 1
            end
        end
    end
    best = x >= p > best ? p : best
    return Int(best)
end

"""
    integer_partitions(n)

List the partitions of the integer `n`.

!!! note
    The order of the resulting array is consistent with that produced by the computational
    discrete algebra software GAP.
"""
function integer_partitions(n::Integer)
    if n < 0
        throw(DomainError(n, "n must be nonnegative"))
    elseif n == 0
        return Vector{Int}[]
    elseif n == 1
        return Vector{Int}[[1]]
    end

    list = Vector{Int}[]

    for p in integer_partitions(n-1)
        push!(list, [p; 1])
        if length(p) == 1 || p[end] < p[end-1]
            push!(list, [p[1:end-1]; p[end]+1])
        end
    end

    list
end

#Noncrossing partitions

if VERSION >= v"0.7.0-DEV.3251"
    const _cmp = cmp
else
    const _cmp = lexcmp
end

#Produces noncrossing partitions of length n
function ncpartitions(n::Int)
    partitions = Vector{Vector{Int}}[]
    _ncpart!(1,n,n,Vector{Int}[], partitions)
    partitions
end

function _ncpart!(a::Int, b::Int, nn::Int, x::Vector, partitions::Vector)
    n = b - a + 1
    for k = 1:n, root in CoolLexCombinations(n, k)
        root .+= a - 1
        #Abort if construction is out of lex order
        !isempty(x) && _cmp(x[end], root) == 1 && return

        #Save if we've filled all the holes
        sofar = Vector{Int}[x..., root]
        ssofaru = sort!(union(sofar...))
        if length(ssofaru) == nn && ssofaru == collect(1:nn)
            push!(partitions, sofar)
            return
        end

        #otherwise patch all remaining holes
        blob = [ssofaru; nn + 1]
        for l = 1:length(blob)-1
            ap, bp = blob[l] + 1, blob[l+1] - 1
            ap <= bp && _ncpart!(ap, bp, nn, sofar, partitions)
        end
    end
end


x = -11:11
y = sin.(x)
x = x[y .> 0]          # Only keep values of x where y=sin x is positive

# Example: Consider the matrix
A = rand(-10:10, 3, 10)

# Compute a boolean array with `true` if the column sums are >=0
pick = sum(A, dims=1) .≥ 0

# Create a new vector with only the columns of A given by the `pick` variable
# Note that since `pick` is a 2D vector (row vector), we use [:] before indexing
B = A[:, pick[:]]


# This file illustrates the calculation of power spectral density and
# total average power of a sinusoidal signal.

using FFTW
using Plots

# Construct the signal
f = 20
fs = 100
l = 2^10
ts = 1 / fs
t = (0 : l -1) * ts
x = cos.(2π * f * t)

# Compute fft
xf = fft(x) * ts
ff = 0 : fs / (l - 1) : fs

# Compute power
psd = abs.(xf).^2 / (l - 1) / ts
Δf = fs / (l - 1)
Pf = sum(psd) * Δf              # Average power calculated in time
Pt = sum(abs.(x).^2) / (l - 1)  # Average power calculated in frequency
@show Pf, Pt

# Plots
plt = plot(layout=1)
plot!(ff, psd, subplot=1)
xlabel!("Frequency [Hz]")
ylabel!("Ψ(f)")
title!("Power Spectal Density")
display(plt)


"Square the sum of the first `n` positive integers"
function square_of_sum(n)
    tmp=0
    for i in 1:n
        tmp+=i
    end
    return tmp^2
end

"Sum the squares of the first `n` positive integers"
function sum_of_squares(n)
    tmp=0
    for i in 1:n
        tmp+=i^2
    end
    return tmp
end

"Subtract the sum of squares from square of the sum of the first `n` positive ints"
function difference(n)
    return abs(sum_of_squares(n) - square_of_sum(n))
end


function absolute_error(y_true::Vector, y_pred::Vector)
    check_args(y_true, y_pred)
    return(abs(y_true .- y_pred))
end

function percent_error(y_true::Vector, y_pred::Vector)
    check_args(y_true, y_pred)
    return((y_true .- y_pred) ./ y_true)
end

function log_error(y_true::Vector, y_pred::Vector)
    check_args(y_true, y_pred)
    return(log(y_true - y_pred))
end

function squared_error(y_true::Vector, y_pred::Vector)
    check_args(y_true, y_pred)
    return((y_true - y_pred) .^ 2)
end

function squared_log_error(y_true::Vector, y_pred::Vector)
    check_args(y_true, y_pred)
    return(log_error(y_true, y_pred) .^ 2)
end

function absolute_percent_error(y_true::Vector, y_pred::Vector)
    check_args(y_true, y_pred)
    return(abs(percent_error(y_true, y_pred)))
end

function mean_error(y_true::Vector, y_pred::Vector)
    check_args(y_true, y_pred)
    return(mean(y_true - y_pred))
end

function mean_absolute_error(y_true::Vector, y_pred::Vector)
    check_args(y_true, y_pred)
    return(mean(absolute_error(y_true, y_pred)))
end

function median_absolute_error(y_true::Vector, y_pred::Vector)
    check_args(y_true, y_pred)
    return(median(absolute_error(y_true, y_pred)))
end

function mean_percent_error(y_true::Vector, y_pred::Vector)
    check_args(y_true, y_pred)
    return(mean(percent_error(y_true, y_pred)))
end

function median_percent_error(y_true::Vector, y_pred::Vector)
    check_args(y_true, y_pred)
    return(median(percent_error(y_true, y_pred)))
end

function mean_squared_error(y_true::Vector, y_pred::Vector)
    check_args(y_true, y_pred)
    return(mean(squared_error(y_true, y_pred)))
end

function median_squared_error(y_true::Vector, y_pred::Vector)
    check_args(y_true, y_pred)
    return(median(squared_error(y_true, y_pred)))
end

function sum_squared_error(y_true::Vector, y_pred::Vector)
    check_args(y_true, y_pred)
    return(sum(squared_error(y_true, y_pred)))
end

function mean_squared_log_error(y_true::Vector, y_pred::Vector)
    check_args(y_true, y_pred)
    return(mean(squared_log_error(y_true, y_pred)))
end

function mean_absolute_percent_error(y_true::Vector, y_pred::Vector)
    check_args(y_true, y_pred)
    return(mean(absolute_percent_error(y_true, y_pred)))
end

function median_absolute_percent_error(y_true::Vector, y_pred::Vector)
    check_args(y_true, y_pred)
    return(median(absolute_percent_error(y_true, y_pred)))
end

function symmetric_mean_absolute_percent_error(y_true::Vector, y_pred::Vector)
    check_args(y_true, y_pred)
    return(mean(abs(y_pred - y_true) / ((abs(y_true) + abs(y_pred)) / 2)))
end

function symmetric_median_absolute_percent_error(y_true::Vector, y_pred::Vector)
    check_args(y_true, y_pred)
    return(median(abs(y_pred - y_true) / ((abs(y_true) + abs(y_pred)) / 2)))
end

function mean_absolute_scaled_error(y_true::Vector, y_pred::Vector)
    check_args(y_true, y_pred)
    n = max(length(y_true), length(y_pred))
    numerator = sum(abs(y_true - y_pred))
    denominator = (n / (n - 1)) * sum(abs(y_true[2:n] - y_pred[1:(n-1)]))
    return(numerator / denominator)
end

function total_variance_score(y_true::Vector, y_pred::Vector)
    check_args(y_true, y_pred)
    return(sum((y_true - mean(y_true)) .^ 2))
end

function explained_variance_score(y_true::Vector, y_pred::Vector)
    check_args(y_true, y_pred)
    return(sum((y_pred - mean(y_true)) .^ 2))
end

function unexplained_variance_score(y_true::Vector, y_pred::Vector)
    check_args(y_true, y_pred)
    return(sum((y_true - y_pred) .^ 2))
end

function r2_score(y_true::Vector, y_pred::Vector)
    check_args(y_true, y_pred)
    return(explained_variance_score(y_true, y_pred) / total_variance_score(y_true, y_pred))
end


# 2520 is the smallest number that can be divided by each of the numbers from 1 to 10
# without any remainder.
# What is the smallest positive number that is evenly
# divisible by all of the numbers from 1 to 20?

# (1,2) (1,3) Means 1 * 2 * 1 * 3
# (2,2) means 2 * 2
include("./primes.jl")
using Primes

function pop(arr, value)
    inx = findfirst(x -> x == value, arr)
    if inx != 0
        deleteat!(arr, inx)
        return value
    end
    return 0
end

function merge(factor_set_1, factor_set_2)
    results = []
    for factor in factor_set_1
        pop(factor_set_2, factor)
        push!(results, factor)
    end

    for factor in factor_set_2
        push!(results, factor)
    end
    return results
end

@assert Primes.prime_factors(2) == [2]
@assert Primes.prime_factors(4) == [2,2]
@assert Primes.prime_factors(8) == [2,2,2]
@assert Primes.prime_factors(21) == [3,7]

@assert merge([2,2], [2,2,3]) == [2,2,3]
@assert merge([2,7], [2,2,3]) == [2,7,2,3]
@assert merge([7], [2,3,3]) == [7,2,3,3]

function min_common_factor(nb)
    factors = map(x -> Primes.prime_factors(x), 2:nb)
    min_factors = reduce(merge, [], factors)
    return prod(min_factors)
end

@assert min_common_factor(10) == 2520
println(min_common_factor(20))


# ------------------------------------------------------------------------------------------
# ## How can we calculate $\pi$?
#
# Given a square of length $2r$, the square's area is
#
# $$A_{square} = (2r)^2 = 4r^2$$
#
# whereas the area of a circle with radius $r$ is
# $$A_{circle} = \pi r^2$$
#
#
#
# Therefore the ratio of the area of the circle to that of the square above is
#
# $$\frac{A_{circle}}{A_{square}} = \frac{\pi r^2}{4r^2} = \frac{\pi}{4}$$
#
# and we can define $\pi$ as
#
# $$\pi = 4\frac{A_{circle}}{A_{square}}$$
#
# This suggests a way to calculate $\pi$: if we have a square and the largest circle that
# fits inside that square, we can determine the ratio of areas of a circle and a square. We
# can calculate this ratio using a monte carlo simulation. We select random points inside a
# square, and we keep track of how often those points also fall inside the circle that fits
# perfectly inside that square.
#
# Given a large enough sampling points, $\frac{A_{circle}}{A_{square}}$ will be equal to the
# fraction of randomly chosen points inside the square that also fall inside the circle.
# Then we can figure out $\pi$!
#
# #### Pseudo-code
#
# Given the above, our algorithm for determining $\pi$ looks like this:
#
# 1. For each of $N$ iterations,
#     1. Select a random point inside a square of area $4r^2$ as Cartesian, $(x, y)$,
# coordinates.
#     1. Determine if the point also falls inside the circle embedded within this square of
# area $\pi r^2$.
#     1. Keep track of whether or not this point fell inside the circle. At the end of $N$
# iterations, you want to know $M$ -- the number of the $N$ random points that fell inside
# the circle!
# 1. Calculate $\pi$ as $4\frac{M}{N}$
#
# #### Exercise
#
# Write a function that calculates $\pi$ using Julia.
#
# The algorithm above should work for any value of $r$ that you choose to use. Make sure you
# make $N$ big enough that the value of $\pi$ is correct to at least a couple numbers after
# the decimal point!
#
# *Hint*:
#
# This will probably be easier if you center your circle and square at the coordinate (0, 0)
# and use a radius of 1. For example, to choose random coordinates within your square at
# position (x, y), you may want to choose x and y so that they are each a value between -1
# and +1. Then any point within a distance of 1 from (0, 0) will fall inside the circle!
#
#
#
#
# #### Solution
#
# In what follows are two functions, `calculate_pi` and `calculate_pi_2`. They use the same
# algorithm to determine `pi` with a couple minor differences in execution. Each of these
# functions is then called to show that as the number of samples, `N`, increases, our value
# for `pi` becomes more precise.
# ------------------------------------------------------------------------------------------

"""
calculate_pi(N = 1000)

Return the value of pi, calculated using a Monte Carlo simulation with N samples. N defaults to 1000.
The radius of the circle, r, defaults to 1.
"""
function calculate_pi(N = 1000)
    how_often_in_circle = 0
    for i in 1:N
        # Generate a random point (x, y) inside the square centered at (0, 0)
        # that has corners at (-1, 1), (-1, 1), (1, -1), and (1, 1)
        x, y = rand([-1, 1])*rand(), rand([-1, 1])*rand()
        # Check if the distance to (x, y) from (0, 0) is less than the radius of 1
        if sqrt(x^2 + y^2) < 1
            how_often_in_circle += 1
        end
    end
    return 4 * how_often_in_circle / N
end

calculate_pi.([10, 100, 1000, 10000, 100_000, 1_000_000_000])

"""
calculate_pi_2(N = 1000)

Return the value of pi, calculated using a Monte Carlo simulation with N samples. N defaults to 1000.
The radius of the circle, r, defaults to 1.
"""
function calculate_pi_2(N = 1000)
    how_often_in_circle = 0
    for i in 1:N
        # Generate a random point (x, y) inside the square centered at (0, 0)
        # that has corners at (-1, 1), (-1, 1), (1, -1), and (1, 1)
        x = 1-2*rand()
        y = 1-2*rand()
        # Check if the distance to (x, y) from (0, 0) is less than the radius of 1
        if x^2 + y^2 < 1
            how_often_in_circle += 1
        end
    end
    return 4 * how_often_in_circle / N
end

calculate_pi.([10, 100, 1000, 10000, 100_000, 1_000_000_000])


@doc raw"""
    RationalQuadraticKernel([α=1 [,β=1]])

The rational-quadratic kernel is given by:

```math
\kappa(\mathbf{x},\mathbf{y})
= \left(1 +\alpha ||\mathbf{x},\mathbf{y}||^2\right)^{-\beta}
\qquad \alpha > 0, \; \beta > 0
```

where ``\alpha`` is a scaling parameter and ``\beta`` is a shape parameter. This kernel can
be seen as an infinite sum of Gaussian kernels. If one sets ``\alpha = \alpha_0 / \beta``,
then taking the limit ``\beta \rightarrow \infty`` results in the Gaussian kernel with
scaling parameter ``\alpha_0``.
"""
struct RationalQuadraticKernel{T<:AbstractFloat} <: MercerKernel{T}
    alpha::HyperParameter{T}
    beta::HyperParameter{T}
    RationalQuadraticKernel{T}(α::Real, β::Real) where {T<:AbstractFloat} = new{T}(
        HyperParameter(convert(T,α), interval(OpenBound(zero(T)), nothing)),
        HyperParameter(convert(T,β), interval(OpenBound(zero(T)), nothing))
    )
end
function RationalQuadraticKernel(α::T1 = 1.0, β::T2 = one(T1)) where {T1<:Real,T2<:Real}
    RationalQuadraticKernel{floattype(T1,T2)}(α, β)
end

@inline rationalquadratickernel(z::T, α::T, β::T) where {T<:AbstractFloat} = (1 + α*z)^(-β)

@inline basefunction(::RationalQuadraticKernel) = SquaredEuclidean()
@inline function kappa(κ::RationalQuadraticKernel{T}, z::T) where {T}
    rationalquadratickernel(z, getvalue(κ.alpha), getvalue(κ.beta))
end


function rectfunc(x::T, x_c::T)::T where T
    if abs(x) <= x_c
        return one(T)
    end

    return zero(T)
end

function raisedcosinefunction(x::T, β, 𝑇)::T where T
    return raisedcosinefunction(x,convert(T,β),convert(T,𝑇))
end

"""
    raisedcosinefunc(x::T, β::T, 𝑇::T)::T
Let low := (1-β)/(2𝑇),
    |x| := abs(x),
    high := (1+β)/(2𝑇).

Returns 1 if |x| ≤ low,
Returns 0.5*(1+cos(π*𝑇/β *(|x|-low))) if low ≤ |x| ≤ high.
Returns 0 otherwise.
"""
function raisedcosinefunction(x::T, β::T, 𝑇::T)::T where T
    two = one(T) + one(T)
    low = (one(T)-β)/(two*𝑇)
    high = (one(T)+β)/(two*𝑇)

    if abs(x) <= low
        return one(T)
    elseif low < abs(x) <= high
        return one(T)/two*( one(T) + cos(π*𝑇/β *(abs(x)-low)) )
    end

    return zero(T)
end

# transition band is cosine. This allows satisfies the power complementary condition.
# cos( (yp-y)/(yp-ys) * π/2 ) for a falling edge. yp is pass band, ys is stop band.
function cosinetransitionlowpassfunction(y::T, yp::T, ys::T)::T where T
    abs_y = abs(y)

    if abs_y <= yp
        return one(T)
    elseif yp < abs_y <= ys
        #return cos( (yp-y)/(ys-yp)*π/2 )
        return cos( (yp-abs_y)/(ys-yp)*π/2 )
    end

    return zero(T)
end

function cosinetransitionhighpassfunction(x::T, xp::T, xs::T)::T where T
    abs_x = abs(x)

    if abs_x <= xs
        return zero(T)
    elseif xs < abs_x <= xp
        #return cos( (xp-x)/(xp-xs)*π/2 )
        return cos( (xp-abs_x)/(xp-xs)*π/2 )
    end

    return one(T)
end

# rs is rising edge's stopband.
# rp is rising edge's passband.
# fp is falling edge's passband.
# fs is falling edge's stopband.
function cosinetransitionbandpassfunction(y::T, rs, rp, fp, fs)::T where T
    abs_y = abs(y)

    # rising edge.
    if abs_y <= rs
        return zero(T)
    elseif rs < abs_y <= rp
        #return cos( (rp-y)/(rp-rs)*π/2 )
        return cos( (rp-abs_y)/(rp-rs)*π/2 )
    end

    # falling edge.
    if rp <= abs_y <= fp
        return one(T)
    elseif fp < abs_y <= fs
        #return cos( (fp-y)/(fs-fp)*π/2 )
        return cos( (fp-abs_y)/(fs-fp)*π/2 )
    end

    return zero(T)
end


# =========================================================================== #
# pdf functions
# =========================================================================== #

function pdf(d::UniformDistribution, x)
    h = 1 / (d.b - d.a)
    if d.a <= x <= d.b
        y = h
    else
        y = 0.0
    end
    return y
end

function pdf(d::NormalDistribution, x)
    y = 1 ./ (sqrt(2*pi) .* d.sigma) .* exp.( -0.5 .* (x .- d.mu).^2 ./ d.sigma.^2 )
    return y
end

function pdf(d::CauchyDistribution, x)
    return 1 ./ ( (pi * d.c) .* (1 .+ ((x .- d.mu) ./ d.c).^2 ) )
end

function pdf(d::MultiNormalDistribution, x)
    n_dim       = length(d.mu)
    det_sigma   = prod( diag(d.L) )
    dx          = x .- d.mu
    arg         = -0.5 .* dot(dx, d.inv_sigma * dx )
    nom         = exp.(arg)
    den         = sqrt( (2*pi)^n_dim * det_sigma )
    return nom ./ den
end


"""
    mandelbrot( c )

Computes the number of evaluations of the following equation before the resulting complex number `z` beginning with `z=c`.
- Input: `c::Number`
- Output: `itrMax::??`

### Questions
- What do you think is the output type of this function?
- Which of the part of the code determines the type?

Sometimes forcing the type of output can be an advantage.
"""
function mandelbrot( c )
    z = c
    itrMax = 80
    for n in 1:itrMax
        if abs(z) > 2
            return n-1
        end
        z = z^2 + c
    end
    return itrMax
end


using NumericalMethodsforEngineers, Test

a = [10. 1. -5.; -20. 3. 20.; 5. 3. 5.]
b = [1., 2., 6.]
n = size(A, 1)
x = 0.0

# Convert to upper triang;e
function to_upper(a::Matrix, b::Vector)
  at = deepcopy(a)
  bt = deepcopy(b)
  r, c = size(at)

  for k in 1:r-1
    if abs(at[k, k]) > 1.0e-6
    for i in k+1:r
      x = at[i, k] / at[k, k]
      at[i, k] = 0.0
      for j in k+1:r
      at[i, j] -= at[k, j] * x
      end
      bt[i] -= bt[k] * x
    end
    else
    md"Zero pivot found in row $k"
    end
  end
  (at, bt)
end

function direct_solve(a::Matrix, b::Vector)
  at = deepcopy(a)
  bt = deepcopy(b)
  r, c = size(at)

  for i in r:-1:1
    x = bt[i]
    if i < r
    for j in i+1:r
      x -= at[i, j] * bt[j]
    end
    end
    bt[i] = x / at[i, i]
  end
  bt
end

println("Modified lhs matrix and rhs")
at, bt = to_upper(a, b)
at |> display
println()
bt |> display
println()

println("Solution vector:")
c_nmfe = direct_solve(at, bt)
c_nmfe |> display
println()

c = a \ [1., 2., 6.]
c |> display

println()
@test ≈(c, c_nmfe, atol=1e-10)
@test round.(a * c; digits=14) == b


"""
    ChirpSignal
Chirp Swept-frequency sine generator.
"""
module ChirpSignal

export chirp

"""
    chirp(T, fs, fl, fh; method="linear", phase=0.0) -> Array{Float64,1}

Chirp Swept-frequency sine generator.
# Arguments
- `T`     : signal's length in second
- `fs`    : sampling rate
- `fl`    : lower frequency at the ending time step, fl <= fs
- `fh `   : higher frequency at the ending time step, fh <= fs
- `phase` : phase of the chirp sigal, like sin(wt + phase)
- `method`: available methods are 'linear','quadratic','exponential' and 'logarithmic'; the default is 'linear'
"""
function chirp(T, fs, fl, fh; method="linear", phase=0.0)
    method == "linear" && return chirpLinear(T, fs, fl, fh; phase=phase)
    method == "quadratic" && return chirpQuadratic(T, fs, fl, fh; phase=phase)
    method == "logarithmic" && return chirpLogarithmic(T, fs, fl, fh; phase=phase)
    method == "exponential" && return chirpExponential(T, fs, fl, fh; phase=phase)
end

function chirpLinear(T, fs, fl, fh; phase=0.0)
    Δt = 1.0/fs;
    fl = min(fs/2,max(0,fl));
    fh = min(fs/2,max(0,fh));
    k  = (fh - fl)/T/2;
    t  = 0:Δt:T-Δt;
    y  = @. sin(6.2831853*(k*t + fl)*t + phase);
end

function chirpQuadratic(T, fs, fl, fh; phase=0.0)
    # f(t) = k * t^2 + fl, k = (fh-fl)/T^2
    # F(t) = k/3 * t^3 + fl*t
    Δt = 1.0/fs;
    fl = min(fs/2,max(0,fl));
    fh = min(fs/2,max(0,fh));
    t  = 0:Δt:T-Δt;
    y  = @. sin(6.2831853*( (fh-fl)/(3*T^2) * t^3 + fl * t) + phase)
end

function chirpLogarithmic(T, fs, fl, fh; phase=0.0)
    # f(t) = k*log(t+1) + fl, k = (fh-fl)/log(T+1)
    # F(t) = k*(t+1)*log(t+1) + (fl-k)*t
    (fl <= fh) || error("fl <= fh not met")
    Δt = 1.0/fs;
    fl = min(fs/2,max(0,fl));
    fh = min(fs/2,max(0,fh));
    t  = 0:Δt:T-Δt;
    k  = (fh-fl)/log(T+1);
    y  = @. sin(6.2831853*( k*(t+1)*log(t+1) + (fl-k)*t ) + phase);
end

function chirpExponential(T, fs, fl, fh; phase=0.0)
    # f(t) = exp(k * t) + (fl-1), k = 1/T*log(fh-fl+1)
    # F(t) = 1/k * exp(k * t) + (fl-1)*t
    (fl <= fh) || error("fl <= fh not met")
    Δt = 1.0/fs;
    fl = min(fs/2,max(0,fl));
    fh = min(fs/2,max(0,fh));
    t  = 0:Δt:T-Δt;
    k  = 1/T*log(fh-fl+1);
    y  = @. sin(6.2831853*( (1/k) * exp(k*t) + (fl-1)*t ) + phase);
end

"""
    chirp(T, fs, f::Function; phase=0.0) -> Array{Float64,1}

Function customizable chirp swept-frequency sine generator.
# Arguments
- `T`     : signal's length in second
- `fs`    : sampling rate
- `f`     : function that defines how the frequency changes vs time
- `phase` : phase of the chirp sigal, like sin(wt + phase)
"""
function chirp(T, fs, f::Function; phase=0.0)
    Δt = 1.0/fs;
    τ  = 0:Δt:T-Δt;
    ϕ  = zeros(length(τ),1);
    x  = zeros(length(τ),1);
    ϕ[1] = f(0)*Δt;
    x[1] = sin(6.2831853*ϕ[1] + phase);
    for t = 2:length(τ)
        ϕ[t] = ϕ[t-1] + f(τ[t-1])*Δt;
        x[t] = sin(6.2831853*ϕ[t] + phase);
    end
    return x
end

end  # module


# ------------------------------------------------------------------------------------------
# Often used in machine learning, a "one hot" vector is a vector of all zeros, except for a
# single `1` entry.
# Representing it as a standard vector is memory-inefficient, so it cries out for a special
# implementation.
# ------------------------------------------------------------------------------------------

struct OneHotVector <: AbstractVector{Int}
    idx::Int
    len::Int
end

Base.size(v::OneHotVector) = (v.len,)

Base.getindex(v::OneHotVector, i::Integer) = Int(i == v.idx)

OneHotVector(3, 10)

A = rand(5,5)

A * OneHotVector(3, 5)

Vector(OneHotVector(3,5))

# ------------------------------------------------------------------------------------------
# ## Exercise
#
# Generalize it to any element type.
# ------------------------------------------------------------------------------------------


using Base.MathConstants

function getHarmonicNumber(n::Integer)::Float64
    H = 0
    for i = 1:n
        H = H + (1 / i)
    end
    return H
end

function getHarmonicNumberWithOrder(n::Integer, order::Integer)::Float64
    H = 0
    for i = 1:n
        H = H + (1 / i^order)
    end
    return H
end

eulermascheroni = convert(Float64, γ)


using DrWatson
@quickactivate "StatReth"

# %%
using Plots
using Distributions

# %%
len = 100
p = range(0, 1, length = len)

# %%
prior = ones(len)
prior ./= sum(prior)

likelihood(w, l) = @. pdf(Binomial(w + l, p), w)

function posterior(prior, likelih)
    post = prior .* likelih
    return post ./ sum(post)
end

posterior_a = posterior(prior, likelihood(3, 0))  # www
posterior_b = posterior(prior, likelihood(3, 1))  # wwwl
posterior_c = posterior(prior, likelihood(5, 2))  # lwwlwww

# %%
plot(xlabel = "p", ylabel = "probability", dpi = 200, legend = :topleft)
plot!(p, posterior_a, label = "a")
plot!(p, posterior_b, label = "b")
plot!(p, posterior_c, label = "c")

# %%
prior = ones(len)
prior[p .< 0.5] .= 0.0
prior ./= sum(prior)

posterior_a = posterior(prior, likelihood(3, 0))  # www
posterior_b = posterior(prior, likelihood(3, 1))  # wwwl
posterior_c = posterior(prior, likelihood(5, 2))  # lwwlwww

# %%
plot(xlabel = "p", ylabel = "probability", dpi = 200, legend = :topleft)
plot!(p, posterior_a, label = "a")
plot!(p, posterior_b, label = "b")
plot!(p, posterior_c, label = "c")


# ------------------------------------------------------------------
# Licensed under the MIT License. See LICENSE in the project root.
# ------------------------------------------------------------------

"""
    signarea(p₁, p₂, p₃)

Compute signed area of triangle formed
by points `p₁`, `p₂` and `p₃`.
"""
function signarea(p₁::Point{2}, p₂::Point{2}, p₃::Point{2})
  a = coordinates(p₁)
  b = coordinates(p₂)
  c = coordinates(p₃)
  ((b[1]-a[1])*(c[2]-a[2]) - (b[2]-a[2])*(c[1]-a[1])) / 2
end

"""
    signarea(triangle)

Compute the signed area of `triangle`.
"""
function signarea(t::Triangle{2})
  vs = vertices(t)
  signarea(vs[1], vs[2], vs[3])
end

"""
    sideof(point, segment)

Determines on which side of the oriented `segment`
the `point` lies. Possible results are `:LEFT`,
`:RIGHT` or `:ON` the segment.
"""
function sideof(p::Point{2,T}, s::Segment{2,T}) where {T}
  a, b = vertices(s)
  area = signarea(p, a, b)
  ifelse(area > atol(T), :LEFT, ifelse(area < -atol(T), :RIGHT, :ON))
end

"""
    sideof(point, chain)

Determines on which side of the closed `chain` the
`point` lies. Possible results are `:INSIDE` or
`:OUTSIDE` the chain.
"""
function sideof(p::Point{2,T}, c::Chain{2,T}) where {T}
  w = windingnumber(p, c)
  ifelse(isapprox(w, zero(T), atol=atol(T)), :OUTSIDE, :INSIDE)
end


# Julia program to Find the Sum of Digits of a Number

# Extracting Each digits
function sum_of_digits(num)
    sum = 0
    while(num != 0)
        sum = sum + (num % 10)
        num = num ÷ 10
    end
    return sum
end

print("Enter the number: ")
num = readline()
num = parse(Int, num)
sum = sum_of_digits(abs(num))
println("The sum of digits of the given number is $sum.")

"""
Time Complexity: O(log(num)), where 'num' is the length of the given number
Space Complexity: O(1)

SAMPLE INPUT AND OUTPUT

SAMPLE 1
Enter the number: -12
The sum of digits of the given number is 3.

SAMPLE 2
Enter the number: 43258
The sum of digits of the given number is 22.

"""


using Plots
using PrettyTables

include("rfiod.jl")
using Main.RFIOD

io = stdout
# Uncomment below to save to file instead
# io = open("output.txt", "w")

F₁(x) = 2x - 3sin(x) + 5
F₁′(x) = 2 - 3cos(x)
F₁root = -2.8832368725582835

F₂(x) = x^3 - 8.5x^2 + 20x - 8
F₂′(x) = 3x^2 - 17x + 20
F₂root = 4

# -- Question 1 -- #

q1plot = plot(F₁, -5:0.1:5, framestyle=:origin, legend = false, title = "F₁(x) = 2x - 3sin(x) + 5 has a single real root in [-5,5]")
# F₁′(x) > 0 outside of this interval with F₁(-5) < 0, F₁(5) > 0, so no roots outside

# -- Binary Search Programming Task -- #
println(io, "-- Binary Search Programming Task --")

# Initial interval [-5, 5]
BSresult1 = binarysearch(F₁, -5, 5)
# Initial interval [-5, 5], but with inputs swapped
BSresult2 = binarysearch(F₁, 5, -5)
# Initial interval [-2.88323688, -2.88323687]
BSresult3 = binarysearch(F₁, -2.88323688, -2.88323687)
# Initial interval [-100, 100]
BSresult4 = binarysearch(F₁, -100, 100)

println(io, "Binary search with initial interval [a,b]")
# Table created for display
tabledata = hcat(
    [-5, 5, -2.88323688, -100],
    [5, -5, -2.88323687, 100],
    Plots.unzip([BSresult1, BSresult2, BSresult3, BSresult4])...
)
pretty_table(io, tabledata, header = ["a", "b", "root", "iterations"])

# Initial interval [-1, 1], which does not contain a root
BSresult5 = try binarysearch(F₁, -1, 1) catch ErrorException "Error, root not found" end
println(io, "Binary search with the invalid initial interval [-1, 1] gives:")
show(io, BSresult5)

# -- Question 3 -- #
println(io, "\n\n-- Question 3 --")

f(k) = x -> x - F₁(x)/(2+k)

"""
    q3FPI(k, x₀, Nₘₐₓ, [ε])

Run the fixed point iteration algorithm used in Question 3, with parameter `k` and starting value `x₀`.

Terminate if consecutive iterations differ by less than `ε`, or after `Nₘₐₓ` iterations.

Outputs a table to the REPL with the results, and returns a staircase/cobweb diagram.

# Examples

```julia-repl
julia> q3FPI(3,-2,10)
Fixed point iteration with k = 3, Nₘₐₓ = 10
┌─────┬──────────┬────────────┬───────────┐
│   n │       xₙ │         ϵₙ │   ϵₙ/ϵₙ₋₁ │
├─────┼──────────┼────────────┼───────────┤
│ 0.0 │     -2.0 │   0.883237 │       0.0 │
│ 1.0 │ -2.74558 │   0.137658 │  0.155857 │
│ 2.0 │ -2.87879 │ 0.00444334 │  0.032278 │
│ 3.0 │ -2.88315 │ 9.00029e-5 │ 0.0202557 │
│ 4.0 │ -2.88324 │ 1.79286e-6 │ 0.0199201 │
│ 5.0 │ -2.88324 │ 3.57019e-8 │ 0.0199133 │
└─────┴──────────┴────────────┴───────────┘
```

"""
function q3FPI(k, x₀, Nₘₐₓ, ε = 1e-5)

    result = fixedpointiteration(f(k), x₀, Nₘₐₓ, ε)
    n = length(result) - 1

    errors = result .- F₁root
    errorratios = [0, (errors[2:(n+1)] ./ errors[1:n])...]
    tabledata = hcat(
        collect(0:n),
        result,
        errors,
        errorratios
    )
    println(io, "Fixed point iteration with k = $k, Nₘₐₓ = $Nₘₐₓ")
    pretty_table(io, tabledata, header = ["n", "xₙ", "ϵₙ", "ϵₙ/ϵₙ₋₁"])

    FPIplot = staircasediagram(f(k), result)
    plot!(FPIplot, title = "k = $k")
    return FPIplot

end

# k = 0, ε = 10⁻⁵, x₀ = -2, Nₘₐₓ = 10
q3plot1 = q3FPI(0, -2, 10)

# k = 4 gives monotonic convergence as f(8)'(x*) > 0
q3plot2 = q3FPI(4, -2, 20)

# k = 2 gives oscillatory convergence as f(1)'(x*) < 0
q3plot3 = q3FPI(2, -2, 20)

# k = 16, Nₘₐₓ = 50
q3FPI(16, -2, 50)

# -- Question 4 -- #
println(io, "\n-- Question 4 --")

g(x) = x - F₂(x)/20

q4result = fixedpointiteration(g, 5, 1000, 1e-5)
# n and xₙ take the final values given below
n = length(q4result) - 1
xₙ = q4result[end]
println(io, "Fixed point iteration to find the double root of
F₂(x) = x³ - 8.5x² + 20x - 8 results in n = $n iterations
with estimated root xₙ = $xₙ")

# -- Question 5 -- #
println(io, "\n-- Question 5 --")

"""
    q5NR(f, x₀, Nₘₐₓ, root, order)

Run the Newton-Raphson algorithm used in Question 5 on the function `f(x) = x - F(x)/F'(x)` starting at `x₀`

Terminate the fixed point iteration after at most Nₘₐₓ iterations.

Outputs a table with errors to `root` and error ratios of order `order`, and returns the vector of iterations.

# Examples

```julia-repl
julia> q5NR(x -> x - sin(x)/cos(x), 3, 10, π, 2)
┌─────┬─────────┬──────────────┬──────────────┐
│   n │      xₙ │           ϵₙ │     ϵₙ/ϵₙ₋₁² │
├─────┼─────────┼──────────────┼──────────────┤
│ 0.0 │     3.0 │    -0.141593 │          0.0 │
│ 1.0 │ 3.14255 │  0.000953889 │    0.0475791 │
│ 2.0 │ 3.14159 │ -2.89316e-10 │ -0.000317963 │
│ 3.0 │ 3.14159 │          0.0 │          0.0 │
│ 4.0 │ 3.14159 │          0.0 │          NaN │
└─────┴─────────┴──────────────┴──────────────┘
5-element Vector{Float64}:
 3.0
 3.142546543074278
 3.141592653300477
 3.141592653589793
 3.141592653589793
```
"""
function q5NR(f, x₀, Nₘₐₓ, root, order)
    result = fixedpointiteration(f, x₀, Nₘₐₓ)
    n = length(result) - 1
    errors = result .- root
    errorratios = [0, (errors[2:(n+1)] ./ errors[1:n].^order)...]
    tabledata = hcat(
        collect(0:n),
        result,
        errors,
        errorratios
    )
    pretty_table(io, tabledata, header = ["n", "xₙ", "ϵₙ", "ϵₙ/ϵₙ₋₁" * (order == 2 ? "²" : "")])
    return result
end

"""
    q5NRgraph(F, result, nlines)
Create a graph from the first `nvalues` iterations of `result`, coming from the Newton-Raphson algorithm on `F`.

# Examples

```julia-repl
julia> q5NRgraph(sin, q5NR(x -> x - sin(x)/cos(x), 3, 10, π, 2), 4)
```
"""
function q5NRgraph(F, result, nlines)
    NRplot = scatter((result[1],F(result[1])), markercolor = :black, framestyle = :origin,
        title = "Newton-Raphson with x₀ = $(result[1])", legend = false)
    for n = 1:nlines
        plot!(NRplot, [result[n],result[n+1],result[n+1]],
            [F(result[n]), 0, F(result[n+1])], linecolor = :black)
    end
    plot!(NRplot, F, linecolor = :orange)
    return NRplot
end

f₁(x) = x - F₁(x)/F₁′(x)
f₂(x) = x - F₂(x)/F₂′(x)

# x₀ = 0 for F₁ gives divergence
println(io, "Newton-Raphson method for \nF₁(x) = 2x - 3sin(x) + 5 with x₀ = 0")
q5result1 = q5NR(f₁, 0, 10, F₁root, 2)
q5plot1 = q5NRgraph(F₁, q5result1, 4)

# x₀ = -2 for F₁ gives convergence
println(io, "Newton-Raphson method for \nF₁(x) = 2x - 3sin(x) + 5 with x₀ = -2")
q5result2 = q5NR(f₁, -2, 10, F₁root, 2)
q5plot2 = q5NRgraph(F₁, q5result2, 2)

# x₀ = 5 for F₂ gives convergence
println(io, "Newton-Raphson method for \nF₂(x) = x³ - 8.5x² + 20x - 8 with x₀ = 5")
q5result3 = q5NR(f₂, 5, 50, F₂root, 1)
q5plot3 = q5NRgraph(F₂, q5result3, 10)

# Saves images automatically, uncomment to use
#=

# File extension e.g. ".png", ".pdf"
ext = ".pdf"

# Makes a new directory for the images (or ignores the error if one exists already)
try
    mkdir("images")
catch IOError
    nothing
end

# Saves the figures created above
try
    savefig(q1plot,"images\\q1-singleroot$ext")
    savefig(q3plot1,"images\\q3-k0-graph$ext")
    savefig(q3plot2,"images\\q3-k4-graph$ext")
    savefig(q3plot3,"images\\q3-k2-graph$ext")
    savefig(q5plot1,"images\\q5-f1divergence-graph$ext")
    savefig(q5plot2,"images\\q5-f1convergence-graph$ext")
    savefig(q5plot3,"images\\q5-f2convergence-graph$ext")
catch IOError
    println("Failed to save images")
end

=#

io == stdout || close(io)
nothing


#md # ---

using DifferentialEquations, Plots

g = 9.79 # Gravitational constants
L = 1.00 # Length of the pendulum

#Initial Conditions
u₀ = [0, π / 60] # Initial speed and initial angle
tspan = (0.0, 6.3) # time domain

#Define the problem
function simplependulum(du, u, p, t)
    θ = u[1]
    dθ = u[2]
    du[1] = dθ
    du[2] = -(g/L)*θ
end

#Pass to solvers
prob = ODEProblem(simplependulum, u₀, tspan)
sol = solve(prob, Tsit5(), reltol = 1e-6)
#md nothing # hide

#md # ---

# Analytic solution
u = u₀[2] .* cos.(sqrt(g / L) .* sol.t)

plot(sol.t, getindex.(sol.u, 2), label = "Numerical")
plot!(sol.t, u, label = "Analytic")
#md savefig("pendulum1.svg"); nothing # hide

#md # ![](pendulum1.svg)

#md # ---

# [Numbers with Uncertainties](http://tutorials.juliadiffeq.org/html/type_handling/02-uncertainties.html)

using Measurements

g = 9.79 ± 0.02; # Gravitational constants
L = 1.00 ± 0.01; # Length of the pendulum

#Initial Conditions
u₀ = [0 ± 0, π / 60 ± 0.01] # Initial speed and initial angle

#Define the problem
function simplependulum(du, u, p, t)
    θ = u[1]
    dθ = u[2]
    du[1] = dθ
    du[2] = -(g/L)*θ
end

#Pass to solvers
prob = ODEProblem(simplependulum, u₀, tspan)
sol = solve(prob, Tsit5(), reltol = 1e-6);
nothing # hide

#md # ---

# Analytic solution
u = u₀[2] .* cos.(sqrt(g / L) .* sol.t)

plot(sol.t, getindex.(sol.u, 2), label = "Numerical")
plot!(sol.t, u, label = "Analytic")
#md savefig("pendulum2.svg"); nothing # hide

#md # ![](pendulum2.svg)


using Statistics

"""
    linear_regression(X::Matrix, Y::Matrix, alpha::Float64, iterations::Int64)

    Linear Regression Algorithm using Gradient Descent
    to find the best fit line for the data.

    Arguments :
        - X : Training data set of shape [m x n]
        - Y : Labels corresponding to the training data set of shape [m x 1]
        - alpha : Learning rate
        - iterations : Number of iterations to train
    Returns :
        - W : Weight vector
        - b : Bias
        - J_history : Cost Function History

    gradient_descent(W::Matrix, b, X::Matrix, Y::Matrix, alpha::Float64, num_iters::Int64)

    Performs gradient descent to learn W and b

    Arguments
        - `W`: matrix of weights
        - `b`: bias vector
        - `X`: matrix of features
        - `Y`: vector of labels
        - `alpha`: learning rate ( generally ranges from 0.01 to 0.00001 )
        - `num_iters`: number of iterations to run gradient descent

    Returns:
        - `W`: matrix of weights
        - `b`: bias vector
        - `J_history`: vector of cost values after each iteration

    # Contributors:- [Navaneeth Sharma](https://github.com/Navaneeth-Sharma)
"""

function gradient_descent(
    W::Matrix,
    b,
    X::Matrix,
    Y::Matrix,
    alpha::Float64,
    num_iters::Int64,
)
    m = length(Y)
    J_history = []
    for i in 1:num_iters
        W = W - alpha .* (1 / m) * (X * (X'W .+ b - Y'))
        b = b - alpha .* (1 / m) * sum((X'W .+ b - Y'))
        push!(J_history, 1 / 2m * sum((X'W .+ b - Y') .^ 2))
    end
    return W, b, J_history
end

function linear_regression(
    X::Matrix,
    Y::Matrix,
    alpha::Float64,
    num_iters::Int64,
)
    W = zeros(1, size(X, 1))
    b = 0
    W, b, J_history = gradient_descent(W, b, X, Y, alpha, num_iters)
    return W, b, J_history
end

# Function to predict the output of a given data vector
predict(X::Matrix, W::Matrix, b::Float64) = X'W .+ b;

# Example usage
X = [1, 3, 2, 5, 7, 8, 8, 9, 10, 12]
Y = [2, 4, 3, 6, 8, 9, 9, 10, 11, 13]

X = reshape(X, 1, length(X));
Y = reshape(Y, 1, length(Y));

W, b, J_history = linear_regression(X, Y, 0.01, 1000);

output = predict(X, W, b)
println("The predicted is: $output")


"A module to plot Ulam spirals"
module UlamSpiral

import PyPlot
using Primes
import Base: position

export position, text_spiral, point_spiral, plot_quadratic

"""
    position(n[, start_n=1])

Compute (x,y) location in the lattice spiral of the integer n, assuming the spiral
starts at the integer start_n (which takes default value 1).

# Arguments
- `n::Integer`: the integer to find the position of, N.B. n >= start_n
- `start_n::Integer=1`: the starting integer of the spiral

# Examples
```jldoctest
julia> UlamSpiral.position(23, 1)
(0, -2)
```
"""
function position( n::Integer, start_n::Integer=1 )
    if n<start_n
        throw(DomainError())
    elseif n==start_n
        return (0, 0)
    end

    j = n-start_n
    k = Int(ceil( floor(sqrt( j )) / 2 )) # how far out in the spiral
    m = Int( j - (2*(k-1)+1)^2 ) # how far along the arm
    length = Int( 2*k )
    quarter = Int( floor(m / length) )
    m2 = m - length*quarter
    if quarter==0
        d_x = k
        d_y =  Int( m2 - length/2 + 1 )
    elseif quarter==1
        d_x = -Int( m2 - length/2 + 1 )
        d_y = k
    elseif quarter==2
        d_x = -k
        d_y = -Int( m2 - length/2 + 1 )
    elseif quarter==3
        d_x =  Int( m2 - length/2 + 1 )
        d_y = -k
    end
    # println(i, ", ", k, ", ", m, ", ", length, ", ", quarter, ", ", d_x, ", ", d_y)
    return (d_x, d_y)
end

"""
    text_spiral( [start_n] )

Write numbers in an Ulam spiral starting from start_n.

# Arguments
- `start_n::Integer=1`: the starting integer of the spiral

# Keyword Arguments
- `fontsize=20`:        the size of the font to write everything
- `color_prime="red"`:  the color to use for prime numbers
- `color_comp="blue"`:  the color to use for composite numbers (note 1 is compositite by convention)
- `width=10`:           the width of the square used (which determines how many numbers to plot)
- `fig_n=1`:            the figure on which to plot everything
- `clear="on"`:         whether to clear the figure before starting

# Examples
```jldoctest
julia> text_spiral()
PyPlot.Figure(PyObject <matplotlib.figure.Figure object at 0x1221f3dd0>)

julia> text_spiral( 100; fontsize=16)
PyPlot.Figure(PyObject <matplotlib.figure.Figure object at 0x1221f3dd0>)

julia> text_spiral(fig_n=2, color_comp="white", color_prime="black", width=20, fontsize=10)
PyPlot.Figure(PyObject <matplotlib.figure.Figure object at 0x122136e10>)

```
"""
function text_spiral(start_n::Integer=1;
                     fontsize=20, color_prime="red", color_comp="blue", width=10, fig_n=1, clear="on")
    c_x = Int(floor(width/2)) # centre of spiral
    c_y = Int(floor(width/2))
    stop_n = start_n + width^2 - 1;

    f = figure(fig_n)
    if clear=="on"
        clf()
    end
    hold(true)
    for n=start_n : stop_n
        (d_x, d_y) = UlamSpiral.position(n, start_n)
        if isprime(n)
            PyPlot.text(c_x + d_x, c_y + d_y, @sprintf("%d", n), fontsize=fontsize, color=color_prime)
        else
            PyPlot.text(c_x + d_x, c_y + d_y, @sprintf("%d", n), fontsize=fontsize, color=color_comp)
         end
    end
    PyPlot.xlim(0, width+1)
    PyPlot.ylim(0, width+1)
    return f
end

"""
    point_spiral( [start_n] )

Plot points at numbers in an Ulam spiral starting from start_n.

# Arguments
- `start_n::Integer=1`: the starting integer of the spiral

# Keyword Arguments
- `marker="o":     the marker to use
- `markersize=7`:  the size of the markers being plotted
- `color="red"`:   the color to use (this routine only plots prime numbers)
- `width=10`:      the width of the square used (which determines how many numbers to plot)
- `fig_n=1`:       the figure on which to plot everything
- `clear="on"`:    whether to clear the previous figure

# Examples
```jldoctest
julia> point_spiral()
PyPlot.Figure(PyObject <matplotlib.figure.Figure object at 0x1221f3dd0>)

julia> point_spiral( 1000; markersize=10)
PyPlot.Figure(PyObject <matplotlib.figure.Figure object at 0x1221f3dd0>)

julia> point_spiral(fig_n=2, markersize=1, color="black", width=200)
PyPlot.Figure(PyObject <matplotlib.figure.Figure object at 0x122136e10>)

julia> text_spiral(fig_n=2, color_comp="white", color_prime="black", width=20, fontsize=10)
julia> point_spiral(fig_n=2, markersize=3, marker="s", color="green", width=20, clear="off")
PyPlot.Figure(PyObject <matplotlib.figure.Figure object at 0x122136e10>)

```
"""
function point_spiral(start_n::Integer=1;
                      marker="o", markersize=7, color="red", width=100, fig_n=1, clear="on")
    c_x = Int(floor(width/2)) # centre of spiral
    c_y = Int(floor(width/2))
    stop_n = start_n + width^2 - 1;

    f = figure(fig_n)
    if clear=="on"
        clf()
    end
    hold(true)
    for n=primes(start_n, stop_n)
        (d_x, d_y) = UlamSpiral.position(n, start_n)
        PyPlot.plot(c_x + d_x, c_y + d_y; marker=marker, markersize=markersize, color=color)
    end
    PyPlot.xlim(0, width+1)
    PyPlot.ylim(0, width+1)
    return f
end

"""
    plot_quadratic(a, b, c)

Highlight points on the quadratic ax^2 + bx + c. Presumes you have already
run point_spiral with the same width parameter.

# Arguments
- `a::Integer = 4`:  quadratic coefficient
- `b::Integer = -2`: linear coefficient
- `c::Integer = 41`: constant coefficient

# Keyword Arguments
- `start_n=1`           the starting integer of the spiral
- `marker="o":          the marker to use
- `markersize=7`:       the size of the markers being plotted
- `color_prime="blue"`: the color to use for prime numbers
- `color_comp="white"`: the color to use for compositive numbers
- `width=10`:              the width of the square used (should be the same as previous plot)
- `fig_n=1`:              the figure on which to plot everything (should be the same as previous plot)

# Examples
```jldoctest
julia> point_spiral(fig_n=2, markersize=1, color="black", width=200);
julia> plot_quadratic(; fig_n=2, width=200);
julia> plot_quadratic(4, 2, 41; fig_n=2, width=200, color_prime="green");
```
"""
function plot_quadratic(a::Integer=4, b::Integer=-2, c::Integer=41;
                        start_n=1, width=10, fig_n=1,
                        marker="o", markersize=7, color_prime="red", color_comp="white")
    c_x = Int(floor(width/2)) # centre of spiral
    c_y = Int(floor(width/2))
    stop_n = start_n + width^2 - 1;
    rad = 0
    x = start_n - 1
    while rad <= width/2
        x = nextprime(x+1)
        n = a*x^2 + b*x + c
        if n>0
            (d_x, d_y) = UlamSpiral.position(n, start_n);
            rad = max( abs(d_x), abs(d_y) )
            plot(c_x + d_x, c_y + d_y, marker=marker, color=color_prime, markersize=markersize)
        end
    end
    # at present not plotting composite numbers at all
end

end # module


module ProjectEuler

include("utils.jl")
using .Utils

function problem001()
    #=
    Problem 1:
        If we list all the natural numbers below 10 that are multiples of 3 or 5,
        we get 3, 5, 6 and 9. The sum of these multiples is 23.
        Find the sum of all the multiples of 3 or 5 below 1000.
    =#
    return sum(i for i in 1:999 if i%3==0 || i%5==0) # => 234168
end

function problem002()
    #=
    Problem 2:
        Each new term in the Fibonacci sequence is generated by
        adding the previous two terms. By starting with 1 and 2,
        the first 10 terms will be:
            1, 2, 3, 5, 8, 13, 21, 34, 55, 89, ...
        By considering the terms in the Fibonacci sequence whose
        values do not exceed four million, find the sum of the
        even-valued terms.
    =#
    sum = 0
    f1,f2 = 1,1
    while f1 <= 4_000_000
        f1, f2 = f1+f2, f1
        if f1 % 2 == 0
            sum += f1
        end
    end
    return sum
end

function problem003()
    #=
    Problem 3:
        The prime factors of 13195 are 5, 7, 13 and 29.
        What is the largest prime factor of the number 600851475143 ?
    =#
    factors = Utils.factorize(600851475143)
    return maximum(i -> i[1],factors) # largest factor
end

function problem004()
    #=
    Problem 4:
        A palindromic number reads the same both ways. The
        largest palindrome made from the product of two 2-digit
        numbers is 9009 = 91 × 99.

        Find the largest palindrome made from the product of two
        3-digit numbers.
    =#
    palindrome_products = (i*j for i=100:999, j=100:999 if string(i*j)==reverse(string(i*j)))
    return maximum(palindrome_products)
end

function problem005()
    #=
    Problem 5:
        2520 is the smallest number that can be divided by each of
        the numbers from 1 to 10 without any remainder.

        What is the smallest positive number that is evenly
        divisible by all of the numbers from 1 to 20?
        =#

        # The desired number will be the product of the factors of
        # each number from 1 to 20, excluding duplicates. For example,
        # 16 = 2^4, so that also covers 2,4,and 8
    return (2^4 * # 2,4,8,16
        3^2 * # 3, 9
        5 *
        7 *
        11 *
        13 *
        17 *
        19
    )
end

function problem006()
    #=
    Problem 6:
        The sum of the squares of the first ten natural numbers is,
            1^2 + 2^2 + ... + 10^2 = 385
        The square of the sum of the first ten natural numbers is,
            (1 + 2 + ... + 10)^2 = 55^2 = 3025
        Hence the difference between the sum of the squares of the
        first ten natural numbers and the square of the sum is
        3025 − 385 = 2640.
        Find the difference between the sum of the squares of the first
        one hundred natural numbers and the square of the sum.
    =#
    return sum(1:100)^2 - sum(i^2 for i=1:100)
end

function problem007()
    #=
    Problem 7:
    By listing the first six prime numbers:
        2, 3, 5, 7, 11, and 13,
    we can see that the 6th prime is 13.
    What is the 10,001'st prime number?
    =#
    for (i,p) in enumerate(Utils.Primes())
        if i == 10001
            return p
        end
    end
end

function problem008()
    #=
    Problem 8
        The four adjacent digits in the 1000-digit number that
        have the greatest product are 9 × 9 × 8 × 9 = 5832.

        73167176531330624919225119674426574742355349194934
        96983520312774506326239578318016984801869478851843
        85861560789112949495459501737958331952853208805511
        12540698747158523863050715693290963295227443043557
        66896648950445244523161731856403098711121722383113
        62229893423380308135336276614282806444486645238749
        30358907296290491560440772390713810515859307960866
        70172427121883998797908792274921901699720888093776
        65727333001053367881220235421809751254540594752243
        52584907711670556013604839586446706324415722155397
        53697817977846174064955149290862569321978468622482
        83972241375657056057490261407972968652414535100474
        82166370484403199890008895243450658541227588666881
        16427171479924442928230863465674813919123162824586
        17866458359124566529476545682848912883142607690042
        24219022671055626321111109370544217506941658960408
        07198403850962455444362981230987879927244284909188
        84580156166097919133875499200524063689912560717606
        05886116467109405077541002256983155200055935729725
        71636269561882670428252483600823257530420752963450

        Find the thirteen adjacent digits in the 1000-digit number
        that have the greatest product. What is the value
        of this product?
    =#

    number_str = """
    73167176531330624919225119674426574742355349194934
    96983520312774506326239578318016984801869478851843
    85861560789112949495459501737958331952853208805511
    12540698747158523863050715693290963295227443043557
    66896648950445244523161731856403098711121722383113
    62229893423380308135336276614282806444486645238749
    30358907296290491560440772390713810515859307960866
    70172427121883998797908792274921901699720888093776
    65727333001053367881220235421809751254540594752243
    52584907711670556013604839586446706324415722155397
    53697817977846174064955149290862569321978468622482
    83972241375657056057490261407972968652414535100474
    82166370484403199890008895243450658541227588666881
    16427171479924442928230863465674813919123162824586
    17866458359124566529476545682848912883142607690042
    24219022671055626321111109370544217506941658960408
    07198403850962455444362981230987879927244284909188
    84580156166097919133875499200524063689912560717606
    05886116467109405077541002256983155200055935729725
    71636269561882670428252483600823257530420752963450
    """
    number = reshape(readdlm(IOBuffer(strip(replace(number_str,""," "))),Int)',1,:) # 1x1000 array
    products = (prod(number[i:(i+12)]) for i in 1:(length(number)-12)) # get product of every rolling 13-digit subarray
    return maximum(products)
end

function problem009()
    #=
    Problem 9:
    A Pythagorean triplet is a set of three natural numbers, a < b < c, for which,
    a^2 + b^2 = c^2
    For example, 3^2 + 4^2 = 9 + 16 = 25 = 5^2.

    There exists exactly one Pythagorean triplet for which a + b + c = 1000.
    Find the product abc.
    =#
end

end # module


# # [Function approximation](@id function-approximation-example)
#
# The objective of this example is to approximate a known function ``f`` by a
# spline.
#
# ## Exact function
#
# We consider the function ``f(x) = e^{-x} \cos(8πx)`` in the interval
# ``x ∈ [0, 1]``.

using CairoMakie
CairoMakie.activate!(type = "svg")

x_interval = 0..1
f(x) = exp(-x) * cospi(8x)

fig = Figure(resolution = (800, 600))
ax = Axis(fig[1, 1]; xlabel = "x")
lines!(ax, x_interval, f)
fig

# ## Approximation space

# To approximate this function using a spline, we first need to define a
# B-spline basis ``\{ b_i \}_{i = 1}^N`` describing a spline space.
# The approximating spline can then be written as
#
# ```math
# g(x) = ∑_{i = 1}^N c_i b_i(x),
# ```
#
# where the ``c_i`` are the B-spline coefficients describing the spline.
# The objective is thus to find the coefficients ``c_i`` that result in the best
# possible approximation of the function ``f``.
#
# Here we use splines of order ``k = 4`` (polynomial degree ``d = 3``, i.e.
# cubic splines).
# For simplicity, we choose the B-spline knots to be uniformly distributed.

using BSplineKit

ξs = range(x_interval; length = 15)
B = BSplineBasis(BSplineOrder(4), ξs)

# We plot below the knots and the basis functions describing the spline space.
# Note that knots are represented by grey crosses.

function plot_knots!(ax, ts; ybase = 0, knot_offset = 0.03, kws...)
    ys = zero(ts) .+ ybase
    ## Add offset to distinguish knots with multiplicity > 1
    if knot_offset !== nothing
        for i in eachindex(ts)[(begin + 1):end]
            if ts[i] == ts[i - 1]
                ys[i] = ys[i - 1] + knot_offset
            end
        end
    end
    scatter!(ax, ts, ys; marker = :x, color = :gray, markersize = 16, kws...)
    ax
end

function plot_basis!(ax, B; eval_args = (), kws...)
    cmap = cgrad(:tab20)
    N = length(B)
    ts = knots(B)
    hlines!(ax, 0; color = :gray)
    for (n, bi) in enumerate(B)
        color = cmap[(n - 1) / (N - 1)]
        i, j = extrema(support(bi))
        lines!(ax, ts[i]..ts[j], x -> bi(x, eval_args...); color, linewidth = 2.5)
    end
    plot_knots!(ax, ts; kws...)
    ax
end

fig = Figure(resolution = (800, 600))
ax = Axis(fig[1, 1]; xlabel = "x", ylabel = "bᵢ(x)")
plot_basis!(ax, B; knot_offset = 0.05)
fig

# ## Approximating the function
#
# Three different methods are implemented in BSplineKit to approximate
# functions.
# In increasing order of accuracy and complexity, these are:
#
# ### 1. [`VariationDiminishing`](@ref)
#
# Implements Schoenberg's variation diminishing approximation.
# This simply consists on estimating the spline coefficients as ``c_i =
# f(x_i)``, where the ``x_i`` are the Greville sites.
# These are obtained by window-averaging the B-spline knots ``t_j``:
#
# ```math
# x_i = \frac{1}{k - 1} ∑_{j = 1}^{k - 1} t_{i + j}.
# ```
#
# This approximation is expected to preserve the shape of the function.
# However, as shown below, it is usually very inaccurate as an actual
# approximation, and should only be used when a qualitative estimation of ``f``
# is sufficient.

S_vd = approximate(f, B, VariationDiminishing())

# ### 2. [`ApproxByInterpolation`](@ref)
#
# Approximates the original function by interpolating on a discrete set of
# interpolation points.
# In other words, the resulting spline exactly matches ``f`` at those points.
#
# By default, the interpolation points are chosen as the Greville sites
# associated to the B-spline basis (using [`collocation_points`](@ref); see also
# [`Collocation.AvgKnots`](@ref)).
# For more control, the interpolation points may also be directly set via the
# [`ApproxByInterpolation`](@ref) constructor.
#
# In the below example, we pass the B-spline basis to the
# `ApproxByInterpolation` constructor, which automatically determines the
# collocation points as explained above.

S_interp = approximate(f, B, ApproxByInterpolation(B))  # or simply approximate(f, B)

# ### 3. [`MinimiseL2Error`](@ref)
#
# Approximates the function by minimising the ``L^2`` distance between ``f`` and
# its spline approximation ``g``.
#
# In other words, it minimises
# ```math
# \mathcal{L}[g] = {\left\lVert f - g \right\rVert}^2 = \left< f - g, f - g \right>,
# ```
# where
# ```math
# \left< u, v \right> = ∫_a^b u(x) \, v(x) \, \mathrm{d}x
# ```
# is the inner product between two functions, and ``a`` and ``b`` are the
# boundaries of the prescribed B-spline basis.
#
# One can show that the optimal coefficients ``c_i`` minimising the ``L^2`` error
# are the solution to the linear system ``\bm{M} \bm{c} = \bm{φ}``,
# where ``M_{ij} = \left< b_i, b_j \right>`` and ``φ_i = \left< b_i, f \right>``.
# These two terms are respectively computed by [`galerkin_matrix`](@ref) and
# [`galerkin_projection`](@ref).
#
# Indeed, this can be shown by taking the differential
# ```math
# δ\mathcal{L}[g] = \mathcal{L}[g + δg] - \mathcal{L}[g]
# = 2 \left< δg, g - f \right>,
# ```
# where ``δg`` is a small perturbation of the spline ``g``.
# The optimal spline ``g^*``, minimising the ``L^2`` distance, is such that
# ``δ\mathcal{L}[g^*] = 0``.
#
# Noting that ``g = c_i b_i`` (where summing is implicitly performed over
# repeated indices), the perturbation is given by ``δg = δc_i b_i``, as the
# B-spline basis is assumed fixed.
# The optimal spline then satisfies
# ```math
# \left< b_i, g^* - f \right> δc_i
# = \left[ \left< b_i, b_j \right> c_j^* - \left< b_i, f \right> \right] δc_i
# = \left[ M_{ij} c_j^* - φ_i \right] δc_i
# = 0
# ```
# for all perturbations ``δ\bm{c}``, leading to the linear system stated above.
#
# As detailed in [`galerkin_projection`](@ref), integrals are computed via
# Gauss--Legendre quadratures, in a way that ensures that the result is exact
# when ``f`` is a polynomial of degree up to ``k - 1`` (or more generally, a
# spline belonging to the space spanned by the chosen B-spline basis).

S_minL2 = approximate(f, B, MinimiseL2Error())

# ## Method comparison
#
# Below, the approximations using the three methods are compared to the actual
# function ``f``.

fig = Figure(resolution = (1000, 750))
colours = theme(fig.scene).palette.color[]
style_vd = (color = colours[3], label = "Variation diminishing")
style_interp = (color = colours[2], label = "Interpolation")
style_minL2 = (color = colours[1], label = "L² minimisation")
let ax = Axis(fig[1:2, 1]; xlabel = "x", ylabel = "Approximation")
    plot_knots!(ax, knots(B); knot_offset = nothing)
    lines!(ax, x_interval, f; color = :black, linewidth = 2, label = "Original")
    lines!(ax, x_interval, x -> S_vd(x); style_vd...)
    lines!(ax, x_interval, x -> S_interp(x); style_interp...)
    lines!(ax, x_interval, x -> S_minL2(x); style_minL2...)
    axislegend(ax)
end
let ax = Axis(fig[1, 2]; ylabel = "Difference with original")
    plot_knots!(ax, knots(B); knot_offset = nothing)
    lines!(ax, x_interval, x -> S_interp(x) - f(x); style_interp...)
    lines!(ax, x_interval, x -> S_minL2(x) - f(x); style_minL2...)
    hidexdecorations!(ax; grid = false)
    axislegend(ax; position = :rt, orientation = :horizontal)
end
let ax = Axis(fig[2, 2]; xlabel = "x", ylabel = "Squared difference", yscale = log10)
    ylims!(1e-8, 1e-2)
    plot_knots!(ax, knots(B); knot_offset = nothing, ybase = 1e-6)
    lines!(ax, x_interval, x -> abs2(S_interp(x) - f(x)); style_interp...)
    lines!(ax, x_interval, x -> abs2(S_minL2(x) - f(x)); style_minL2...)
end
fig

# As seen above, the **variation diminishing approximation**, while capturing the
# shape of the original function, doesn't really provide an accurate
# approximation of it.
#
# The other two methods are much more accurate.
# On the right half of the figure, a detailed comparison of the two is provided,
# by plotting the difference between each approximation and the actual ``f``
# function.
#
# The **interpolation method** works pretty well, matching exactly the actual
# function at the interpolation points.
# Note that, in this example, most interpolation points match the spline knots.
# This is because we're using splines of even degree (``k = 4``) and because
# knots are uniformly spaced.
#
# Nevertheless, when looking at the global error, the **``L^2`` minimisation
# method** works best, as expected.
# In particular, as seen above, it reduces the maximum approximation error (i.e.
# the ``L^∞`` distance, ``{\left\lVert f - g \right\rVert}_∞ = \max |f(x) -
# g(x)|``) compared to the interpolation approach.


struct Point
    x::Float64
    y::Float64
end

function ccw(a::Point, b::Point, c::Point)
    return ((b.x - a.x)*(c.y - a.y) - (b.y - a.y)*(c.x - a.x))
end

function graham_scan!(points::Vector{Point})
    N = length(points)

    # Place the lowest point at the start of the array
    sort!(points, by = item -> item.y)

    # Sort all other points according to angle with that point
    other_points = sort(points[2:end], by = item -> atan(item.y - points[1].y,
                                                         item.x - points[1].x))

    # Place points sorted by angle back into points vector
    for i in 1:length(other_points)
        points[i+1] = other_points[i]
    end

    # M will be the point on the hull
    M = 2
    for i = 1:N
        while (ccw(points[M-1], points[M], points[i]) <= 0)
            if (M > 2)
                M -= 1
            # All points are collinear
            elseif (i == N)
                break
            else
                i += 1
            end
        end

        # ccw point found, updating hull and swapping points
        M += 1
        points[i], points[M] = points[M], points[i]
    end

    return points[1:M]
end

function main()
    # This hull is just a simple test so we know what the output should be
    points = [
        Point(-5.,2), Point(5,7), Point(-6,-12), Point(-14,-14), Point(9,9),
        Point(-1,-1), Point(-10,11), Point(-6,15), Point(-6,-8), Point(15,-9),
        Point(7,-7), Point(-2,-9), Point(6,-5), Point(0,14), Point(2,8)
    ]
    hull = graham_scan!(points)
    println(hull)
end

main()


## Exercise 4-6
## Write a function called circle that takes a turtle, t, and radius, r, as parameters and that draws an approximate circle by calling polygon with an appropriate length and number of sides. Test your function with a range of values of r.

## TIP
## Figure out the circumference of the circle and make sure that len * n == circumference.
using ThinkJulia

println("Ans: ")

function polygon(turtle::Turtle, nsides::Int, len::Int = 100)
    angle = (360 / nsides)

    for i in 1:nsides
        forward(turtle, len)
        turn(turtle, -angle)
    end
end

function circle(turtle::Turtle, radius::Int)
    circumference = 2 * π * radius
    n = Int(trunc(circumference ÷ 3) + 3)
    len = Int(trunc(circumference / n))
    @show n, len
    polygon(turtle, n, len)
end

@svg begin
    turtle = Turtle()
    circle(turtle, 100)
end

println("End.")


"""
        entropy(decisionMat)

Apply Entropy method for a given matrix and criteria types.

# Arguments:
 - `decisionMat::DataFrame`: n × m matrix of objective values for n alternatives and m criteria

# Description
entropy() applies the Entropy method to calculate objective weights which are obtained through multi-step calculations of the decision matrix constructed from the actual information about the evaluation criteria of the alternatives.

# Output
- `::EntropyResult`: EntropyResult object that holds multiple outputs including weighting and best index.

# Examples
```julia-repl

julia> df = DataFrame(
           C1 = [2, 4, 3, 5, 4, 3],
           C2 = [1, 1, 2, 1, 2, 2],
           C3 = [4, 5, 6, 5, 5, 6],
           C4 = [7, 6, 6, 7, 6, 6],
           C5 = [6, 7, 5, 6, 7, 6],
           C6 = [6, 7, 6, 7, 7, 6],
           C7 = [7, 6, 8, 7, 6, 6],
           C8 = [3000, 3500, 4000, 3000, 3000, 3500]
           )
6×8 DataFrame
 Row │ C1     C2     C3     C4     C5     C6     C7     C8
     │ Int64  Int64  Int64  Int64  Int64  Int64  Int64  Int64
─────┼────────────────────────────────────────────────────────
   1 │     2      1      4      7      6      6      7   3000
   2 │     4      1      5      6      7      7      6   3500
   3 │     3      2      6      6      5      6      8   4000
   4 │     5      1      5      7      6      7      7   3000
   5 │     4      2      5      6      7      7      6   3000
   6 │     3      2      6      6      6      6      6   3500

julia> result = entropy(df);

julia> result.w
8-element Array{Float64,1}:
 0.29967360959745126
 0.441367338923551
 0.07009088719659533
 0.021238237112072383
 0.0490229289460051
 0.023080378850660263
 0.0477633096868323
 0.0477633096868323

julia> result.bestIndex
2
```
# References

Shannon, C. E. (1948). A Mathematical Theory of Communication. Bell System Technical Journal, 27(3), 379–423. doi:10.1002/j.1538-7305.1948.tb01338.x
Ulutaş, A . (2019). Entropi ve MABAC Yöntemleri ile Personel Seçimi. OPUS Uluslararası Toplum Araştırmaları Dergisi, 13 (19), 1552-1573. DOI: 10.26466/opus.580456.

"""
function entropy(decisionMat::DataFrame):EntropyResult

    row, col = size(decisionMat)
    normalizeDM = zeros(Float64, row, col)

    for i in 1:col
        normalizeDM[:, i] = decisionMat[:,i] ./ sum(decisionMat[:, i])
    end

    logMat = zeros(Float64, row, col)
    for i in 1:row
        for j in 1:col
            logMat[i,j] = normalizeDM[i, j] .* log(normalizeDM[i, j])
        end
    end

    e = zeros(Float64, col)

    for i in 1:col
        e[i] = 1 - sum(logMat[:,i]) ./ -log(row)
    end

    w = zeros(Float64, col)

    for i in 1:col
        w[i] = e[i] ./ sum(e)
    end

    rankings = sortperm(w)

    bestIndex = rankings |> last

    result = EntropyResult(
        decisionMat,
        w,
        rankings,
        bestIndex
    )

    return result
end


"""
Family of nonlinear transformation parametrized by λ that defaults to Box Cox
"""
struct nonlinearTransform{T1<:Function, T2<:Function, T3<:Function, T4<:Function, T5<:Function, T6<:Function, T7<:Function}
    f::T1
    df::T2
    d2f::T3
    df_hyp::T4
    d2f_hyp::T5
    d_mixed::T6
    inv::T7
end

@doc raw"""
    BoxCox(λ::Real)

The Box-Cox power transformation function with parameter λ.

```math
g_\lambda(x) = \begin{cases}
    \frac{x^\lambda - 1}{\lambda} & \lambda \neq 0\\
    \ln(x) & \lambda = 0
\end{cases}
```

External links
* [Power Transform on Wikipedia](https://en.wikipedia.org/wiki/Power_transform)

...
# Arguments
* `x::float64`: evaluation point
* `lambda::float=1.0`: lambda hyperparameter

"""
function boxCox(x, lambda=1)
    lambda == 0 ? Base.log.(x) : (float(x).^lambda.-1)./lambda
end

"""
Derivative of Box-Cox power transformation w.r.t x
"""
function boxCoxPrime(x, lambda=1)
    lambda==0 ? float(x).^(-1) : float(x).^(lambda .-1)
end

"""
Second derivative of Box-Cox power transformation w.r.t x
"""
function boxCoxPrime2(x, lambda=1)
    lambda==0 ? -float(x).^(-2) : (lambda-1)*float(x).^(lambda-2)
end

"""
Derivative of Box-Cox power transformation w.r.t lambda
"""
function boxCoxPrime_lambda(x, lambda=1)
    lambda==0 ? 0 : (lambda * float(x).^lambda .* log.(x) .- float(x).^lambda .+ 1)/lambda^2
end

"""
Second derivative of Box-Cox power transformation w.r.t lambda
"""
function boxCoxPrime_lambda2(x, lambda=1)
    lambda==0 ? 0 : float(x).^lambda .* (Base.log.(x)^2)/lambda -
    (2*x.^lambda .* Base.log.(x))/lambda^2 + 2*(float(x).^lambda-1)/lambda^3
end

"""
Mixed derivative of Box-Cox power transformation w.r.t lambda and x
"""
function boxCoxMixed_lambda_z(x, lambda=1)
    lambda==0 ? 0 : float(x).^(lambda-1) .* Base.log.(x)
end

"""
Inverse Box Cox power transformation
"""
function invBoxCox(y, lambda=1)
    lambda==0 ? Base.exp.(y) : Base.exp.(Base.log.(lambda.*y.+1)./lambda)
end

#define BoxCox Object
boxCoxObj = nonlinearTransform(boxCox, boxCoxPrime, boxCoxPrime2, boxCoxPrime_lambda, boxCoxPrime_lambda2, boxCoxMixed_lambda_z, invBoxCox)

@doc raw"""
    TODO Unimplimented
"""
struct YeoJohnson{T} <: Function end

@doc raw"""
    TODO Unimplimented
"""
struct ArandaOrdaz{T} end


"""
PDE_Tools

This file has the operators I need for the PDE example. They
live in a separate file to make the CI easier for me to organize.
"""
# Famous sparse matrices
"""
Dx2d(n)

returns x partial on n x n grid.
Unit square, homogeneous Dirichlet BC
"""
function Dx2d(n)
    h = 1 / (n + 1)
    ssdiag = ones(n^2 - 1) / (2 * h)
    for iz = n:n:n^2-1
        ssdiag[iz] = 0.0
    end
    updiag = Pair(1, ssdiag)
    lowdiag = Pair(-1, -ssdiag)
    Dx = spdiagm(lowdiag, updiag)
    return Dx
end

"""
Dy2d(n)

returns y partial on n x n grid.
Unit square, homogeneous Dirichlet BC
"""
function Dy2d(n)
    h = 1 / (n + 1)
    ssdiag = ones(n^2 - n) / (2 * h)
    updiag = Pair(n, ssdiag)
    lowdiag = Pair(-n, -ssdiag)
    Dy = spdiagm(lowdiag, updiag)
    return Dy
end

"""
Lap2d(n)

returns the negative Laplacian in two space dimensions
on n x n grid.

Unit square, homogeneous Dirichlet BC
"""
function Lap2d(n)
    # hm2=1/h^2
    hm2 = (n + 1.0)^2
    maindiag = fill(4 * hm2, (n^2,))
    sxdiag = fill(-hm2, (n^2 - 1,))
    sydiag = fill(-hm2, (n^2 - n,))
    for iz = n:n:n^2-1
        sxdiag[iz] = 0.0
    end
    D2 = spdiagm(-n => sydiag, -1 => sxdiag, 0 => maindiag, 1 => sxdiag, n => sydiag)
    return D2
end

"""
u=fish2d(f, fdata)

Fast Poisson solver in two space dimensions.
Same as the Matlab code.
Unit squre + homogeneous Dirichlet BCs.

Grid is nx by nx

You give me f as a two-dimensional vector f(x,y).
I return the solution u.
"""
function fish2d(f, fdata)
    u = fdata.utmp
    v = fdata.uhat
    T = fdata.T
    ST = fdata.ST
    (nx, ny) = size(f)
    nx == ny || error("need a square grid in fish2d")
    u .= f
    u = ST * u
    u = u'
    u1 = reshape(u, (nx * nx,))
    v1 = reshape(v, (nx * nx,))
    v1 .= u1
    ldiv!(u1, T, v1)
    u = u'
    u .= ST * u
    u ./= (2 * nx + 2)
    return u
end

"""
fishinit(n)

Run FFTW.plan_r2r to set up the solver. Do not mess
with this function.
"""
function fishinit(n)
    #
    # Get the sine transform from FFTW. This is faster/better/cleaner
    # than what I did in the Matlab codes.
    #
    zstore = zeros(n, n)
    ST = FFTW.plan_r2r!(zstore, FFTW.RODFT00, 1)
    uhat = zeros(n, n)
    fishu = zeros(n, n)
    TD = newT(n)
    T = lu!(TD)
    fdata = (ST = ST, uhat = uhat, utmp = zstore, T = T, fishu = fishu)
    return fdata
end

"""
T = newT(n)

Builds the n^2 x n^2 sparse tridiagonal matrix for
the 2D fast Poisson solver.
"""
function newT(n)
    N = n * n
    h = 1 / (n + 1)
    x = h:h:1-h
    h2 = 1 / (h * h)
    LE = 2 * (2 .- cos.(pi * x)) * h2
    fn = ones(N - 1) * h2
    gn = ones(N - 1) * h2
    dx = zeros(N)
    for k = 1:n-1
        fn[k*n] = 0.0
        gn[k*n] = 0.0
        dx[(k-1)*n+1:n*k] = LE[k] * ones(n)
    end
    dx[(n-1)*n+1:n*n] = LE[n] * ones(n)
    T = Tridiagonal(-fn, dx, -gn)
    return T
end

"""
Use fish2d and reshape for preconditioning.
"""
function Pfish2d(v, fdata)
    n2 = length(v)
    n = Int(sqrt(n2))
    (n * n == n2) || error("input to Pfish2d not a square array")
    v2 = reshape(v, (n, n))
    u = fish2d(v2, fdata)
    u = reshape(u, (n2,))
    return u
end

"""
Pvec2d(v, u, pdata)

Returns inverse Laplacian * v

u is a dummy argument to make nsoli happy

Preconditioner for nsoli
"""
function Pvec2d(v, u, pdata)
    fdata = pdata.fdata
    p = Pfish2d(v, fdata)
    return p
end


# Huber loss function

using LinearAlgebra

export HuberLoss

"""
**Huber loss**

    HuberLoss(ρ=1, μ=1)

Returns the function
```math
f(x) = \\begin{cases}
    \\tfrac{μ}{2}\\|x\\|^2 & \\text{if}\\ \\|x\\| ⩽ ρ \\\\
    ρμ(\\|x\\| - \\tfrac{ρ}{2}) & \\text{otherwise},
\\end{cases}
```
where `ρ` and `μ` are positive parameters.
"""
struct HuberLoss{R <: Real, S <: Real} <: ProximableFunction
    rho::R
    mu::S
    function HuberLoss{R, S}(rho::R, mu::S) where {R <: Real, S <: Real}
        if rho <= 0 || mu <= 0
            error("parameters rho and mu must be positive")
        else
            new(rho, mu)
        end
    end
end

is_convex(f::HuberLoss) = true
is_smooth(f::HuberLoss) = true

HuberLoss(rho::R=1, mu::S=1) where {R <: Real, S <: Real} = HuberLoss{R, S}(rho, mu)

function (f::HuberLoss)(x::AbstractArray{T}) where {R <: Real, T <: Union{R, Complex{R}}}
    normx = norm(x)
    if normx <= f.rho
        return f.mu / R(2) * normx^2
    else
        return f.rho * f.mu * (normx - f.rho / R(2))
    end
end

function gradient!(y::AbstractArray{T}, f::HuberLoss, x::AbstractArray{T}) where {R <: Real, T <: Union{R, Complex{R}}}
    normx = norm(x)
    if normx <= f.rho
        y .= f.mu .* x
        v = f.mu / R(2) * normx^2
    else
        y .= (f.mu * f.rho) / normx .* x
        v = f.rho * f.mu * (normx - f.rho / R(2))
    end
    return v
end

function prox!(y::AbstractArray{T}, f::HuberLoss, x::AbstractArray{T}, gamma::R=R(1)) where {R <: Real, T <: Union{R, Complex{R}}}
    normx = norm(x)
    mugam = f.mu*gamma
    scal = (1 - min(mugam / (1 + mugam), mugam * f.rho / normx))
    for k in eachindex(y)
        y[k] = scal*x[k]
    end
    normy = scal*normx
    if normy <= f.rho
        return f.mu / R(2) * normy^2
    else
        return f.rho * f.mu * (normy - f.rho / R(2))
    end
end

fun_name(f::HuberLoss) = "Huber loss"
fun_dom(f::HuberLoss) = "AbstractArray{Real}, AbstractArray{Complex}"
fun_expr(f::HuberLoss) = "x ↦ (μ/2)||x||² if ||x||⩽ρ, μρ(||x||-ρ/2) otherwise"
fun_params(f::HuberLoss) = string("ρ = $(f.rho), μ = $(f.mu)")

function prox_naive(f::HuberLoss, x::AbstractArray{T}, gamma::R=R(1)) where {R <: Real, T <: Union{R, Complex{R}}}
    y = (1 - min(f.mu * gamma / (1 + f.mu * gamma), f.mu * gamma * f.rho / norm(x))) * x
    if norm(y) <= f.rho
        return y, f.mu / R(2) * norm(y)^2
    else
        return y, f.rho * f.mu * (norm(y) - f.rho / R(2))
    end
end


"""
    RMSE(obs,sim)

    Returns the Root Mean Squared Error between observations `obs` and simulations `sim`.

    The closer to 0 the better.
"""
function RMSE(obs, sim)
    return sqrt(sum((obs .- sim).^2) / length(obs))
end

"""
    NRMSE(obs,sim)

    Returns the Normalized Root Mean Squared Error between observations `obs` and simulations `sim`.
    Normalization is performed using division by observations range (max-min).

    Output: Float/Particles
"""
function NRMSE(obs, sim)
    return sqrt(sum((obs .- sim).^2) / length(obs)) / (findmax(obs)[1] - findmin(obs)[1])
end

"""
    EF(obs,sim)

    Returns the Efficiency Factor between observations `obs` and simulations `sim` using NSE (Nash-Sutcliffe efficiency) model.
    More information can be found at https://en.wikipedia.org/wiki/Nash%E2%80%93Sutcliffe_model_efficiency_coefficient.

    The closer to 1 the better.
"""
function EF(obs, sim)
    SSres = sum((obs - sim).^2)
    SStot = sum((obs .- mean(obs)).^2)
    return 1 - SSres / SStot
end

"""
    dr(obs,sim)

    Returns the Willmott’s refined index of agreement dᵣ.
    Willmot et al. 2011. A refined index of model performance. https://rmets.onlinelibrary.wiley.com/doi/10.1002/joc.2419

    The closer to 1 the better.

"""
function dr(obs, sim)
    a = sum(abs.(obs .- sim))
    b = 2 * sum(abs.(obs .- mean(obs)))
    return 0 + (1 - a / b) * (a <= b) + (b / a - 1) * (a > b)
end


#!/usr/bin/env julia

# Project 1: Vanilla options in a Black-Scholes World
#
# Page 437+, Joshi's Mathematical Finance

# Best to run this with JULIA_NUM_THREADS=$(nproc) and julia 1.3

# Notation: if f is a function of x, fv is one realisation of that function

import ThreadsX
using SpecialFunctions: erf

export C, P, B, present_value, rand_price, mc_pricer, binary, mc_pricer_pathdep

N(x) = 0.5 * (1 + erf(x/sqrt(2)))

d_1(S,t,T,K,r,σ) = (log(S/K) + (r + (σ^2)/2)*(T-t)) / (σ*√(T-t))
d_2(S,t,T,K,r,σ) = (log(S/K) + (r - (σ^2)/2)*(T-t)) / (σ*√(T-t))

# Price of a call option (i.e. the holder has the right to buy stock at maturity)
# S = current stock (underlying) price
# t = current time (time since option created)
# T = time of maturity
# K = the price at which the stock may be bought at maturity
# r = the continuously compounded risk-free rate
# σ = volatility of the underlying (standard deviation per root unit of time)
# TODO: d = dividend rate
function C(S,t; T=1, K=100, r=0.02, σ=0.5)
    d_1v = d_1(S,t,T,K,r,σ)
    d_2v = d_2(S,t,T,K,r,σ)
    N(d_1v)*S - N(d_2v)*present_value(K,r,t,T)
end

# Price of a put option (i.e. holder may sell at maturity)
function P(S,t; T=1, K=100, r=0.02, σ=0.5)
    present_value(K,r,t,T) - S + C(S,t, T=T, K=K, r=r, σ=σ)
end

# Price of a binary option:
# Payoff of 1 if S(T) > E, 0 otherwise
#
# Source: http://www.iam.fmph.uniba.sk/institute/stehlikova/fd14en/lectures/05_black_scholes_1.pdf
#
# Can construct a less than binary thusly:
# less_than_binary = present_value(1) - greater_than_binary
# (cf. put call parity)
function binary(S,t; T=1, E=100, r=0.02, σ=0.5)
    present_value(1,r,t,T) * N(d_2(S,t,T,E,r,σ))
end

##############################
#                            #
# Validation via Monte Carlo #
#                            #
##############################

# Brownian motion
B(T;B0=100,r=0.02,d=0.00,σ=0.5) = B0*exp((r-d)*T-0.5*σ^2*T+σ*√T*randn())

# Sketch of pricer: Brownian motion generates a final stock price; payoff of option is calculated and then discounted. Price of option is average of these.

present_value(v,r,t,T) = v*exp(-r*(T-t))

function rand_price(payoff;S_0=100,t=0,T=1,K=S_0,r=0.02,σ=0.05)
    S = B(T-t;B0=S_0,r=r,d=0,σ=σ)
    present_value(payoff(S),r,t,T)
end

# Strictly we should use a thread-safe RNG here
function mc_pricer(
    payoff;S_0=100,t=0,T=1,K=S_0,r=0.02,σ=0.05, trials=100_000_000
)
    linear, squared = ThreadsX.mapreduce(
        x->begin
            p = rand_price(payoff;S_0=S_0,t=t,T=T,K=K,r=r,σ=σ)
            (p,p^2) # Second term is to keep track of standard error
        end,
        (a,b)->a.+b,
        Base.OneTo(trials),
        init=(0.0, 0.0)
    )

    mean = linear/trials

    (mean = mean, sem = sqrt(squared/trials - mean^2)/sqrt(trials))
end

function mc_pricer_pathdep(
    payoff;S_0=100,t=[0],T=1,K=S_0,r=0.02,σ=0.05, trials=100_000_000
)
    linear, squared = ThreadsX.mapreduce(
        x->begin
            p = B.(T.-t;B0=S_0,r=r,d=0,σ=σ)
            price = present_value(payoff(p),r,t[1],T) # Payoff must operate on time series
            (price,price^2)
        end,
        (a,b)->a.+b,
        Base.OneTo(trials),
        init=(0.0, 0.0)
    )

    mean = linear/trials

    (mean = mean, sem = sqrt(squared/trials - mean^2)/sqrt(trials))
end


"""
    diffper(n,xspan)

Construct 2nd-order differentiation matrices for functions with
periodic end conditions, using `n` unique nodes in the interval
`xspan`. Returns a vector of nodes and the matrices for the first
and second derivatives.
"""
function diffper(n,xspan)
    a,b = xspan
    h = (b-a)/n
    x = @. a + h*(0:n-1)   # nodes, omitting the repeated data

    # Construct Dx by diagonals, then correct the corners.
    dp = fill(0.5/h,n-1)        # superdiagonal
    dm = fill(-0.5/h,n-1)       # subdiagonal
    Dx = diagm(-1=>dm,1=>dp)
    Dx[1,n] = -1/(2*h)
    Dx[n,1] = 1/(2*h)

    # Construct Dxx by diagonals, then correct the corners.
    d0 =  fill(-2/h^2,n)        # main diagonal
    dp =  ones(n-1)/h^2         # superdiagonal and subdiagonal
    Dxx = diagm(-1=>dp,0=>d0,1=>dp)
    Dxx[1,n] = 1/(h^2)
    Dxx[n,1] = 1/(h^2)

    return x,Dx,Dxx
end

"""
    parabolic(ϕ,xspan,m,g₁,g₂,tspan,init)

Solve a parabolic PDE by the method of lines. The PDE is
∂u/∂t = `ϕ`(t,x,u,∂u/∂x,∂^2u/∂x^2), `xspan` gives the space
domain, m gives the degree of a Chebyshev spectral discretization,
`g₁` and `g₂` are functions of (u,∂u/∂x) at the domain ends that
should be made zero, `tspan` is the time domain, and `init` is a
function of x that gives the initial condition. Returns a vector
`x` and a function of t that gives the semidiscrete solution at `x`.
"""
function parabolic(ϕ,xspan,m,g₁,g₂,tspan,init)
    x,Dₓ,Dₓₓ = diffcheb(m,xspan)
    int = 2:m    # indexes of interior nodes

    function extend(v)
        function objective(ubc)
            u₀,uₘ = ubc
            uₓ = Dₓ*[u₀;v;uₘ]
            return [g₁(u₀,uₓ[1]),g₂(uₘ,uₓ[end])]
        end
        ubc = levenberg(objective,[0,0])[end]
        return [ubc[1];v;ubc[2]]
    end

    function ode!(f,v,p,t)
        u = extend(v)
        uₓ,uₓₓ = Dₓ*u,Dₓₓ*u
        @. f = ϕ(t,x[int],u[int],uₓ[int],uₓₓ[int])
    end

    ivp = ODEProblem(ode!,init.(x[int]),float.(tspan))
    u = solve(ivp)

    return x,t->extend(u(t))
end