# ---
# jupyter:
#   jupytext:
#     formats: jl:light,ipynb
#     text_representation:
#       extension: .jl
#       format_name: light
#       format_version: '1.5'
#       jupytext_version: 1.5.1
#   kernelspec:
#     display_name: Julia 1.4.2
#     language: julia
#     name: julia-1.4
# ---

# # Tensor Manipulation: Psi4 and NumPy manipulation routines
# Contracting tensors together forms the core of the Psi4Julia project. First let us consider the popluar [Einstein Summation Notation](https://en.wikipedia.org/wiki/Einstein_notation) which allows for very succinct descriptions of a given tensor contraction.
#
# For example, let us consider a [inner (dot) product](https://en.wikipedia.org/wiki/Dot_product):
# $$c = \sum_{ij} A_{ij} * B_{ij}$$
#
# With the Einstein convention, all indices that are repeated are considered summed over, and the explicit summation symbol is dropped:
# $$c = A_{ij} * B_{ij}$$
#
# This can be extended to [matrix multiplication](https://en.wikipedia.org/wiki/Matrix_multiplication):
# \begin{align}
# \rm{Conventional}\;\;\;  C_{ik} &= \sum_{j} A_{ij} * B_{jk} \\
# \rm{Einstein}\;\;\;  C &= A_{ij} * B_{jk} \\
# \end{align}
#
# Where the $C$ matrix has *implied* indices of $C_{ik}$ as the only repeated index is $j$.
#
# However, there are many cases where this notation fails. Thus we often use the generalized Einstein convention. To demonstrate let us examine a [Hadamard product](https://en.wikipedia.org/wiki/Hadamard_product_(matrices)):
# $$C_{ij} = \sum_{ij} A_{ij} * B_{ij}$$
#
#
# This operation is nearly identical to the dot product above, and is not able to be written in pure Einstein convention. The generalized convention allows for the use of indices on the left hand side of the equation:
# $$C_{ij} = A_{ij} * B_{ij}$$
#
# Usually it should be apparent within the context the exact meaning of a given expression.
#
# Finally we also make use of Matrix notation:
# \begin{align}
# {\rm Matrix}\;\;\;  \bf{D} &= \bf{A B C} \\
# {\rm Einstein}\;\;\;  D_{il} &= A_{ij} B_{jk} C_{kl}
# \end{align}
#
# Note that this notation is signified by the use of bold characters to denote matrices and consecutive matrices next to each other imply a chain of matrix multiplications!

# ## Tensor Operations
#
# To perform most operations we turn to tensor packages (here we use [TensorOperations.jl](https://github.com/Jutho/TensorOperations.jl)). Those allow Einstein convention as an input. In addition to being much easier to read, manipulate, and change, it has (usually) optimal performance.
# First let us import our normal suite of modules:

using PyCall: pyimport
psi4 = pyimport("psi4")
np   = pyimport("numpy")
using TensorOperations: @tensor

# We can then use conventional Julia loops or `@tensor` to perform the same task.

using BenchmarkTools: @btime, @belapsed

# With `@btime`/`@belapsed` we average time over several executions to have more reliable timings than `@time`/`@elapsed` (single execution).

# **WARNING: We are using Julia's global variables, and those are known to be less efficient than local variables. It is better to wrap code inside function.**

# To begin let us consider the construction of the following tensor (which you may recognize):
# $$G_{pq} = 2.0 * I_{pqrs} D_{rs} - 1.0 * I_{prqs} D_{rs}$$
#
# Keep size relatively small as these 4-index tensors grow very quickly in size.

# +
dims = 20

@assert dims <= 30 "Size must be smaller than 30."
D = rand(dims, dims)
I = rand(dims, dims, dims, dims)

# Build the Fock matrix using loops, while keeping track of time
println("Time for loop G build:")
Gloop = @btime begin
   Gloop = np.zeros((dims, dims))
   @inbounds for ind in CartesianIndices(I)
       p, q, r, s = Tuple(ind)
       Gloop[p, q] += 2I[p, q, r, s] * D[r, s]
       Gloop[p, q] -=  I[p, r, q, s] * D[r, s]
   end
   Gloop
end

# Build the Fock matrix using einsum, while keeping track of time
println("Time for @tensor G build:")
G = @btime @tensor G[p,q] := 2I[p,q,r,s] * D[r,s] - I[p,r,q,s] * D[r,s]

# Make sure the correct answer is obtained
println("Loop and einsum builds of the Fock matrix match?    ", np.allclose(G, Gloop))
println()
# Print out relative times for explicit loop vs einsum Fock builds
#println("G builds with einsum are $(g_loop_time/einsum_time) times faster than Julia loops!")
# -

# As you can see, the `@tensor` macro can be considerably faster than plain Julia loops.

# ## Matrix multiplication chain/train
#
# Now let us turn our attention to a more canonical matrix multiplication example such as:
# $$D_{il} = A_{ij} B_{jk} C_{kl}$$
#
# Matrix multiplication is an extremely common operation in all branches of linear algebra. Thus, these functions have been optimized to be extremely efficient. `@tensor` uses it. The matrix product will explicitly compute the following operation:
# $$C_{ij} = A_{ij} * B_{ij}$$
#
#
# This is Julia's matrix multiplication method `*` for matrices.

# +
dims = 200
A = rand(dims, dims)
B = rand(dims, dims)
C = rand(dims, dims)

# First compute the pair product
tmp_dot = A * B
@tensor tmp_tensor[i,k] := A[i,j] * B[j,k]
println("Pair product allclose? ", np.allclose(tmp_dot, tmp_tensor))
# -

# Now that we have proved exactly what `*` product does, let us consider the full chain and do a timing comparison:

D_dot = A * B * C
@tensor D_tensor[i,l] := A[i,j] * B[j,k] * C[k,l]
println("Chain multiplication allclose? ", np.allclose(D_dot, D_tensor))

# +
println()
println("* time:")
@btime A * B * C

println()
println("@tensor time:")
@btime @tensor D_tensor[i,l] := A[i,j] * B[j,k] * C[k,l];
# -

# Both have similar timings, and both call [Basic Linear Algebra Subprograms (BLAS)](https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms). The BLAS routines are highly optimized and threaded versions of the code.
#  - The `@tensor` code will factorize the operation by default; Thus, the overall cost is not ${\cal O}(N^4)$ (as there are four indices) rather it is the factored $(\bf{A B}) \bf{C}$ which runs ${\cal O}(N^3)$.
#
# Therefore you do not need to factorize the expression yourself (sometimes you might need):

println("@tensor factorized time:")
@btime @tensor begin
   tmp[i,k]  := A[i,j] * B[j,k]
   tmp2[i,l] := tmp[i,k] * C[k,l]
end
nothing

# On most machines the three have similar timings. The BLAS usage is usually recommended. Thankfully, in Julia its syntax is very clear. The Psi4Julia project tends to lean toward usage of tensor packages but if Julia's built-in matrix multiplication is significantly cleaner/faster we would use it. The real value of tensor packages will become tangible for more complicated expressions.

# ## Complicated tensor manipulations
# Let us consider a popular index transformation example:
# $$M_{pqrs} = C_{pi} C_{qj} I_{ijkl} C_{rk} C_{sl}$$
#
# Here, a naive loop implementation would scale like $\mathcal{O}(N^8)$ which translates to an extremely costly computation for all but the smallest $N$. A smarter implementation (factorizing the whole expression) would scale
# like $\mathcal{O}(N^5)$.

# **WARNING: First execution is slow because of compilation time. Successive are more honest to the running time.**

# +
# Grab orbitals
dims = 15
@assert dims <= 15 || "Size must be smaller than 15."

C = rand(dims, dims)
I = rand(dims, dims, dims, dims)

# @tensor full transformation.
print("\nStarting @tensor full transformation...")
n8_time = @elapsed @tensor MO_n8[I,J,K,L] := C[p,I] * C[q,J] * I[p,q,r,s] * C[r,K] * C[s,L]
print("complete in $n8_time s\n")

# @tensor factorized N^5 transformation.
print("\nStarting @tensor factorized N^5 transformation with einsum ... ")
n5_time = @elapsed @tensor begin
   MO_n5[A,q,r,s] := C[p,A] * I[p,q,r,s]
   MO_n5[A,B,r,s] := C[q,B] * MO_n5[A,q,r,s]
   MO_n5[A,B,C,s] := C[r,C] * MO_n5[A,B,r,s]
   MO_n5[A,B,C,D] := C[s,D] * MO_n5[A,B,C,s]
end
print("complete in $n5_time s \n")
println("  @tensor factorized is $(n8_time/n5_time) faster than full @tensor algorithm!")
println("  Allclose? ", np.allclose(MO_n8, MO_n5))

# Julia's GEMM N^5 transformation.
# Try to figure this one out!
print("\nStarting Julia's factorized transformation with * ... ")
dgemm_time = @elapsed begin
   MO = C' * reshape(I, dims, :)
   MO = reshape(MO, :, dims) * C
   MO = permutedims(reshape(MO, dims, dims, dims, dims), (2, 1, 4, 3))

   MO = C' * reshape(MO, dims, :)
   MO = reshape(MO, :, dims) * C
   MO = permutedims(reshape(MO, dims, dims, dims, dims),(2, 1, 4, 3))
end
print("complete in $dgemm_time s \n")
println("  * factorized is $(n8_time/dgemm_time) faster than full @tensor algorithm!")
println("  Allclose? ", np.allclose(MO_n8, MO))

# There are still several possibilities to explore:
# @inbounds, @simd, LinearAlgebra.LAPACK calls, Einsum.jl, Tullio.jl, ...
# -
# None of the above algorithms is $\mathcal{O}(N^8)$. `@tensor` factorizes the expression to achieve better performance. There is a small edge in doing the factorization manually. Factorized algorithms have similar timings, although it is clear that with `@tensor` is easier than with Julia's built-in `*`. To use the usual matrix multiplication with tensors we have to reshape and permute their dimensions, subtracting appeal to the simple `*` syntax.


"""
    forwardsub(L,b)

Solve the lower-triangular linear system with matrix `L` and
right-hand side vector `b`.
"""
function forwardsub(L,b)
    n = size(L,1)
    x = zeros(n)
    x[1] = b[1]/L[1,1]
    for i in 2:n
        s = sum( L[i,j]*x[j] for j in 1:i-1 )
        x[i] = ( b[i] - s ) / L[i,i]
    end
    return x
end

"""
    backsub(U,b)

Solve the upper-triangular linear system with matrix `U` and
right-hand side vector `b`.
"""
function backsub(U,b)
    n = size(U,1)
    x = zeros(n)
    x[n] = b[n]/U[n,n]
    for i in n-1:-1:1
        s = sum( U[i,j]*x[j] for j in i+1:n )
        x[i] = ( b[i] - s ) / U[i,i]
    end
    return x
end

"""
    lufact(A)

Compute the LU factorization of square matrix `A`, returning the
factors.
"""
function lufact(A)
    n = size(A,1)        # detect the dimensions from the input
    L = diagm(ones(n))   # ones on main diagonal, zeros elsewhere
    U = zeros(n,n)
    Aₖ = float(copy(A))  # make a working copy

    # Reduction by outer products
    for k in 1:n-1
        U[k,:] = Aₖ[k,:]
        L[:,k] = Aₖ[:,k]/U[k,k]
        Aₖ -= L[:,k]*U[k,:]'
    end
    U[n,n] = Aₖ[n,n]
    return LowerTriangular(L),UpperTriangular(U)
end

"""
    plufact(A)

Compute the PLU factorization of square matrix `A`, returning the
triangular factors and a row permutation vector.
"""
function plufact(A)
    n = size(A,1)
    L = zeros(n,n)
    U = zeros(n,n)
    p = fill(0,n)
    Aₖ = float(copy(A))

    # Reduction by outer products
    for k in 1:n-1
        p[k] = argmax(abs.(Aₖ[:,k]))
        U[k,:] = Aₖ[p[k],:]
        L[:,k] = Aₖ[:,k]/U[k,k]
        Aₖ -= L[:,k]*U[k,:]'
    end
    p[n] = argmax(abs.(Aₖ[:,n]))
    U[n,n] = Aₖ[p[n],n]
    L[:,n] = Aₖ[:,n]/U[n,n]
    return LowerTriangular(L[p,:]),U,p
end


# alias for equivalent uniform measures
# TODO: If we generalize Haar for G-manifolds, then we can add Haar under left-action of O(n)
# here.
const UniformStiefelMeasures{n,k,P} = Hausdorff{Stiefel{n,k,ℝ},P}

# In general, given matrix z ∈ ℝ^(n × k) with IID std normal elements, p=z(z'z)^(-1/2) is drawn
# from the Hausdorff measure on St(n,k).
# This implementation uses the unique QR decomposition z=QR where R[i,i] > 0, where Q is then
# drawn from the Hausdorff measure on St(n,k).
# See Theorem 2.3.19 of Gupta AK, Nagar DK. Matrix variate distributions. CRC Press; 2018
function Base.rand(rng::AbstractRNG, ::Type, μ::Hausdorff{<:Stiefel{n,k,ℝ}}) where {n,k}
    p = randn!(rng, similar(μ.point))
    Q, R = qr(p)
    p .= Matrix(Q) .* sign.(view(R, diagind(R))')
    return p
end

function MeasureTheory.logdensity(
    ::UniformStiefelMeasures{n,k}, ::UniformStiefelMeasures{n,k}, x::AbstractMatrix
) where {n,k}
    return zero(eltype(x))
end

# Chikuse, 2003 Eq. 1.4.8
volume(::Stiefel{n,k,ℝ}) where {n,k} = 2^k * π^(k * n//2) / mvgamma(k, n//2)


@doc raw"""
    Sphere

An unconstrained implementation of the Sphere function defined as:

```math
f(\mathbf{x}) = \sum_{i=1}^{d} x_i^2
```

where ``d`` is the dimension of the input vector ``\mathbf{x}``.
"""
struct Sphere <: Unconstrained end

_sphere(x) = sum(x.^2)

@doc raw"""
    Easom

An unconstrained implementation of the 2-dimensional
Easom function defined as:

```math
f(\mathbf{x}) = -\cos{(x_1)} \cos{(x_2)} \exp{[-(x_1 - \pi)^2 - (x_2 - \pi)^2]}
```

where ``x_1`` and ``x_2`` refer to the first and second element of the
input vector ``\mathbf{x}``.
"""
struct Easom <: Unconstrained end

@inline function _easom(x)
    @assert length(x) == 2 "This is a 2D function"

    term_1 = -cos(x[1]) * cos(x[2])
    term_2 = exp(-(x[1] - π)^2 - (x[2] - π)^2)

    return term_1 * term_2
end

@doc raw"""
    Ackley

An unconstrained implementation of the d-dimensional
Ackley function defined as:

```math
f(\mathbf{x}) = -20 e^{ -0.02 \sqrt{\frac{1}{d}\sum_{i=1}^{d}{x_i^2}}} - e^{\frac{1}{d}\sum_{i=1}^{d}{\cos{(2 \pi x_i)}}} + 20 + e
```

where ``d`` is the dimension of the input vector ``\mathbf{x}``.
"""
struct Ackley <: Unconstrained end

@inline function _ackley(x)
    dimension = length(x)
    @assert dimension > 0 "Must have positive dimension"

    term_1 = exp(-0.02 * sqrt(_sphere(x) / dimension))
    term_2 = exp(sum(cos.(2.0 * π * x)) / dimension)

    return -20.0 * term_1 - term_2 + 20.0 + exp(1.0)
end

@doc raw"""
    Rosenbrock

An unconstrained implementation of the d-dimensional
Rosenbrock function defined as:

```math
f(\mathbf{x}) = \sum_{i=1}^{N-1} \left[100(x_{i-1}-x_i^2)^2 +(1-x_i)^2 \right]
```

where ``N`` is the dimension of the input vector ``\mathbf{x}``.
"""
struct Rosenbrock <: Unconstrained end

@inline function _rosenbrock(x)
    dimension = length(x)
    @assert dimension >= 2 "Must be at least 2D"

    total = 0.0
    for i in 1:dimension - 1
        @inbounds total += 100.0 * (x[i + 1] - x[i]^2)^2 + (1.0 - x[i])^2
    end

    return total
end  # function _rosenbrock

@doc raw"""
    Goldstein-Price

An unconstrained implementation of the d-dimensional
Goldstein-Price function defined as:

```math
f(x,y)=[1 + (x + y + 1)^2(19 − 14x+3x^2− 14y + 6xy + 3y^2)] \times \\
[30 + (2x − 3y)^2(18 − 32x + 12x^2 + 4y − 36xy + 27y^2)]
```
"""
struct GoldsteinPrice <: Unconstrained end

@inline function _goldprice(x)
    @assert length(x) == 2 "Exactly 2D"

    term_1 = (x[1] + x[2] + 1.0)^2
    term_1 *= 19.0 - 14.0 * x[1] + 3.0 * x[1]^2 - 14.0 * x[2] + 6.0 * x[1] * x[2] + 3.0 * x[2]^2
    term_1 += 1.0
    term_2 = (2.0 * x[1] - 3.0 * x[2])^2
    term_2 *= 18.0 - 32.0 * x[1] + 12.0 * x[1]^2 + 48.0 * x[2] - 36.0 * x[1] * x[2] + 27.0 * x[2]^2
    term_2 += 30.0

    return term_1 * term_2
end  # function _goldprice

@doc raw"""
    Beale

An unconstrained implementation of the d-dimensional
Beale function defined as:

```math
f(x, y) = (1.5-x+xy)^2+(2.25-x+xy^2)^2+(2.625-x+xy^3)^2
```
"""
struct Beale <: Unconstrained end

@inline function _beale(x)
    @assert length(x) == 2 "Exactly 2D"

    term_1 = (1.5 - x[1] + x[1] * x[2])^2
    term_2 = (2.25 - x[1] + x[1] * x[2]^2)^2
    term_3 = (2.625 - x[1] + x[1] * x[2]^3)^2

    return term_1 + term_2 + term_3
end  # function _beale

@doc raw"""
    Lévy

An unconstrained implementation of the d-dimensional
Lévy function defined as:

```math
f(\mathbf{x}) = \sin^{2}{\pi w_1} + \sum_{i=1}^{d-1} (w_i-1)^2 [1+10\sin^{2}{\pi w_1 + 1}]
+ (w_d-1)^2 [1+\sin^{2}{2\pi w_d}]
```

where

``w_i = 1 + \frac{x_i-1}{4}`` and ``d`` is the dimension of the vector.
"""
struct Levy <: Unconstrained end

@inline function _levy(x)
    ω = @. 1.0 + (x - 1.0) / 4.0

    term_1 = (sin(π * ω[1]))^2

    term_2 = sum(@. (ω - 1.0)^2 * (1.0 + 10.0 * (sin(π * ω + 1.0))^2))
    term_3 = (ω[end] - 1.0)^2 * (1.0 + (sin(2.0 * π * ω[end]))^2)

    return term_1 + term_2 + term_3
end  # function _levy

# Build a dictionary of test functions and their implementations
test_functions = Dict([
    :Sphere => :_sphere,
    :Easom => :_easom,
    :Ackley => :_ackley,
    :Rosenbrock => :_rosenbrock,
    :GoldsteinPrice => :_goldprice,
    :Beale => :_beale,
    :Levy => :_levy
])

for (k, v) in test_functions
    # Create the methods for the given test functions
    @eval $k(x::T) where T = $v(x)
    # Create a special method for the `TestFunction` type
    @eval evaluate(b::$k, x::T) where T = $v(x)
end


### A Pluto.jl notebook ###
# v0.12.20

using Markdown
using InteractiveUtils

# ╔═╡ 5590424a-6d0b-11eb-13a5-ed80e8342a06
begin
	using Test
	using LinearAlgebra
end

# ╔═╡ 1d03f5fe-6d10-11eb-1918-c1478c31b083
md"""
[Exam pdf link](https://ocw.mit.edu/courses/mathematics/18-085-computational-science-and-engineering-i-fall-2008/exams/quiz1_18085f07.pdf)
"""

# ╔═╡ 0cac1572-6d10-11eb-2699-f1d5f5baa4e3
md"""
## \#1
"""

# ╔═╡ 67a6401a-6d0b-11eb-1803-756ee33e062d
begin
	A₀ = [
		-1  1  0 0
		 0 -1  1 0
		 0  0 -1 1
	]

	A₀ᵀ = A₀'

	A₀ᵀA₀ = A₀ᵀ * A₀

	@test A₀ᵀA₀ == [
	  1  -1   0   0
	 -1   2  -1   0
	  0  -1   2  -1
	  0   0  -1   1
	]

	@test A₀ᵀA₀ * [
		1
		1
		1
		1
	] == [
		0
		0
		0
		0
	]

	@test det(A₀ᵀA₀) == 0

	A₁ = [
		 1  0 0
		-1  1 0
		 0 -1 1
	]

	A₁ᵀA₁ = A₁' * A₁

	@test A₁ᵀA₁ == [
		 2  -1   0
 		-1   2  -1
  		 0  -1   1
	]

	@test det(A₁ᵀA₁) == 1
end

# ╔═╡ 3fb30142-6d10-11eb-2a9c-939b9a40a2da
md"""
## \#2
"""

# ╔═╡ 41e88e3c-6d10-11eb-27ea-254fd36ce6cb
begin
	B = [
		 1 -1  0
		-1  2 -1
		 0 -1  1
	]

	values, Q = eigen(B)

	Λ = diagm(0 => values)

	@test B == round.(Int, Q * Λ * Q')
end

# ╔═╡ Cell order:
# ╠═5590424a-6d0b-11eb-13a5-ed80e8342a06
# ╟─1d03f5fe-6d10-11eb-1918-c1478c31b083
# ╠═0cac1572-6d10-11eb-2699-f1d5f5baa4e3
# ╠═67a6401a-6d0b-11eb-1803-756ee33e062d
# ╠═3fb30142-6d10-11eb-2a9c-939b9a40a2da
# ╠═41e88e3c-6d10-11eb-27ea-254fd36ce6cb


function bracket_sign_change(f′, a, b; k=2)
    if a > b; a,b = b,a; end # ensure a < b
    center, half_width = (b+a)/2, (b-a)/2
    while f′(a)*f′(b) > 0
        half_width *= k
        a = center - half_width
        b = center + half_width
    end
    return (a,b)
end

function bisection(f′, a, b, ϵ)
    if a > b; a,b = b,a; end # ensure a < b

    ya, yb = f′(a), f′(b)
    if ya == 0; b = a; end
    if yb == 0; a = b; end

    while b - a > ϵ
        x = (a+b)/2
        y = f′(x)
        if y == 0
            a, b = x, x
        elseif sign(y) == sign(ya)
            a = x
        else
            b = x
        end
    end
    return (a,b)
end

f = x->sin(10x)+cos(3x)
df(x) = 10*cos(10*x)-3*sin(3*x)

as,bs = bracket_sign_change(df,3.4,3.8)
as,df(as),bs,df(bs)

af,bf = bisection(df,as,bs,1.0e-8)

df(af),f(af)


function get_points_spherical_random(n::Int)
    r = ones(n)
    θ = acos.(1-2rand(n))
    ϕ = 2pi*rand(n)

    return [r θ ϕ]
end

function get_points_spherical_evenly(n::Int)
    r = ones(n)
    θ = acos.(1-2linspace(0.0, 1.0, n))
    ϕ = 2pi*linspace(0.0, 1.0, n)

    return [r θ ϕ]
end

function get_points_cartesian_random(n::Int)
    points = get_points_spherical_random(n)

    return [sin.(points[:,2]).*cos.(points[:,3]) sin.(points[:,2]).*sin.(points[:,3]) cos.(points[:,2])]
end

function random_sphere_pack(r::AbstractFloat, R::AbstractFloat, n::Int)

    step = r/10
    x = [ collect(-step : -step : -R); collect(0 : step : R)]
    y = [ collect(-step : -step : -R); collect(0 : step : R)]
    z = [ collect(-step : -step : -R); collect(0 : step : R)]
    len = length(x)
    places = zeros(len, len, len)

    for i = 1:len
        for j = 1:len
            for k = 1:len
                if sqrt(x[i]^2 + y[j]^2 + z[k]^2) < R
                    places[i,j,k] = 1
                end
            end
        end
    end

    points = zeros(3, n)
    count = 0

    while count < n
        vacant_points = find(places)
        if isempty(vacant_points)
            error("there are no more vacant points in the space of the cluster at `count`=$count")
        end

        random_vacant_point = vacant_points[rand(1:length(vacant_points))]
        (I,J,K) = ind2sub(places, random_vacant_point)

        if places[I,J,K] != 1
            display("Something went wrong! places(p(1),p(2),p(3))!=1")
        end

        for i = 1:len
            for j = 1:len
                for k = 1:len
                    if sqrt((x[i]-x[I])^2 + (y[j]-y[J])^2 + (z[k]-z[K])^2) < 2r
                        places[i,j,k] = 0
                    end
                end
            end
        end

        count += 1;
        points[:, count] = [x[I] y[J] z[K]]
    end

    return points
end


# # Relaxed Two-Way Partitioning Problem
#
# We consider the (nonconvex) two-way partitioning problem from Boyd and Vandenberghe (2004), p.219 \[1\]:
# $$
# \begin{array}{ll} \text{minimize} &  x^\top W x \\
# \text{subject to} &  x_i^2 = 1, \quad i=1,\dots, n,
# \end{array}
# $$
# with $x \in \mathbf{R}^n$ and $W \in \mathbf{S}^n$. The problem can be interpreted as finding a partition of the points $i = 1,\dots, n$ into two sets, where the cost of two points in the same set is $W_{ij}$ and $-W_{ij}$ otherwise. Brute forcing the solution $x^*$ takes $2^n$ attempts and becomes quickly intractable, e.g. for $n \geq 30$. However, a lower-bound for this problem can be computed by solving the convex dual of the problem:
# $$
# \begin{array}{ll} \text{maximize} &  -\sum_i \nu \\
# \text{subject to} &  W + \text{diag}(\nu) \in \mathbf{S}_+^n.
# \end{array}
# $$
# Solving this problem with optimal objective $d^*$ yields a lower bound as least as good as a lower bound based on the minimum eigenvalue $d^* \geq \lambda_{\text{min}}(W)$.
# Let's set up the problem for $n = 20$ and with a specially structured (banded) $W$:

#-
using LinearAlgebra, Random, COSMO, JuMP, IterTools

rng = Random.MersenneTwister(212313);
n = 20;

W = diagm(0 => randn(rng, n));
W += diagm(1 => randn(rng, n-1));
W = Symmetric(W)

# As you can see, the matrix $W$ imposes a structure on the LMI-constraint. Accordingly, COSMO will be able to use chordal decomposition to decompose the LMI constraint into multiple smaller constraints. This can make a significant difference in the performance of the algorithm for large $n$.

#-

model = JuMP.Model(COSMO.Optimizer);
@variable(model, ν[1:n]);
@objective(model, Max, -ones(n)' * ν )
@constraint(model, Symmetric(W + diagm(ν) )  in JuMP.PSDCone());
JuMP.optimize!(model)

# Looking at the solver output you can see how the PSD constraint was decomposed into 19 PSD constraints. Let's look at the lower bound:
JuMP.objective_value(model)

# As $n$ is small, we can verify our result by finding the optimal solution by trying out all possible combinations:

function brute_force_optimisation(W, n)
   opt_obj = Inf
   opt_x = Inf * ones(n)

   for xt in Iterators.product([[1.; -1.] for k = 1:n]...)
      x = [i for i in xt]
      obj_val = x' * W * x
      if obj_val < opt_obj
         opt_obj = obj_val
         opt_x = x
      end
   end
   return opt_obj, opt_x
end
opt_obj, opt_x = brute_force_optimisation(W, n)
# ## References
# [1] Boyd and Vandenberghe - Convex Optimization, Cambridge University Press (2004)


function C(n, k)
    if k == 0 || k == n
        return 1
    else
        return C(n - 1, k) + C(n-1, k-1)
    end
end

function get_basic_Bernstein(n, x)
    coeffs = []
    for k in 1:n
        push!(coeffs, C(n, k) * x^k * (1 - x)^(n - k))
    end
    return coeffs
end

function Bernstein_polynom(f::Function, n, x)
    coeffs = get_basic_Bernstein(n,x)
    result = 0
    for k in 1:n
        result += f(k/n) * coeffs[k]
    end
    return result
end

function Bernstein_polynom(f, n, x)
    coeffs = get_basic_Bernstein(n, x)
    result = 0
    for k in 1:n
        result += coeffs[k]*f[k]
    end
    return result
end

function test_Bernstein()
    test1 = []
    test2 = []
    for i in 0:100
        k = i / 100
        push!(test1, 2^k)
        push!(test2, Bernstein_polynom(x->2^x, 30, k))
    end
    plot(test1)
    plot(test2)
end


# ---
# title: 231. Power of Two
# id: problem231
# author: Tian Jun
# date: 2020-10-31
# difficulty: Easy
# categories: Math, Bit Manipulation
# link: <https://leetcode.com/problems/power-of-two/description/>
# hidden: true
# ---
#
# Given an integer `n`, return _`true` if it is a power of two. Otherwise,
# return `false`_.
#
# An integer `n` is a power of two, if there exists an integer `x` such that `n
# == 2x`.
#
#
#
# **Example 1:**
#
#
#
#     Input: n = 1
#     Output: true
#     Explanation: 20 = 1
#
#
# **Example 2:**
#
#
#
#     Input: n = 16
#     Output: true
#     Explanation: 24 = 16
#
#
# **Example 3:**
#
#
#
#     Input: n = 3
#     Output: false
#
#
# **Example 4:**
#
#
#
#     Input: n = 4
#     Output: true
#
#
# **Example 5:**
#
#
#
#     Input: n = 5
#     Output: false
#
#
#
#
# **Constraints:**
#
#   * `-231 <= n <= 231 - 1`
#
#
## @lc code=start
using LeetCode

function is_power_of_two(n::Int)
    return n > 0 && (n & (n - 1)) == 0
end
## @lc code=end


"""
    plot(f [; iterations])

Plot the graph of the \$n\$-th iterative of a function
\$f:\\mathbb{R}\\rightarrow\\mathbb{R}\$.

#### Arguments
- `f::Function`: Function \$f:\\mathbb{R}\\rightarrow\\mathbb{R}\$.
- `iterations::Int`: Number of iterations to calculate the iterative \$f^n\$.
"""
function plot(f::Function; iterations::Int=1)
    SDDGraphics.supported(:drawlinesegment)

    # Verifying functions
    @assert typeof(f(1.)) <: Real

    SDDGraphics.newdrawing()
    #SDDGraphics.color(SDDGraphics.fgcolor())

    x1, x2 = SDDGraphics.xlims()
    w = SDDGraphics.width()
    Δx = (x2-x1)/w

    xi0 = x1
    yi0 = xi0
    for n in 1:iterations
        yi0 = f(yi0)
    end

    for i in 1:w
        xi1 = x1 + i*Δx
        yi1 = xi1
        for n in 1:iterations
            yi1 = f(yi1)
        end
        SDDGraphics.drawlinesegment(xi0,yi0,xi1,yi1)
        xi0, yi0 = xi1, yi1
    end

    SDDGraphics.drawing()
end

"""
    graphicalanalysis(f, x0 [; iterations])

Plot the graph of a function \$f:\\mathbb{R}\\rightarrow\\mathbb{R}\$ and
the the graphical analysis of the orbit of \$x_0\$.

#### Arguments
- `f::Function`: Function \$f:\\mathbb{R}\\rightarrow\\mathbb{R}\$.
- `x0::Real`: Initial point for the orbit.
- `iterations::Int`: Number of iterations to calculate the orbit of \$x_0\$.
"""
function graphicalanalysis(f::Function, x0::Real; iterations::Int=10)
    SDDGraphics.supported(:drawlinesegment)

    # Verifying functions
    @assert typeof(f(1.)) <: Real

    SDDGraphics.newdrawing()

    x1, x2 = SDDGraphics.xlims()
    y1, y2 = SDDGraphics.ylims()
    w = SDDGraphics.width()
    Δx = (x2-x1)/w

    # Plot of the function's graph
    #SDDGraphics.color(SDDGraphics.fgcolor())
    xi0 = x1
    yi0 = f(x1)
    for i in 1:w
        xi1 = x1 + i*Δx
        yi1 = f(xi1)
        SDDGraphics.drawlinesegment(xi0,yi0,xi1,yi1)
        xi0, yi0 = xi1, yi1
    end

    # Plot the graph of the identity
    SDDGraphics.color(RGB(0.5,0.5,0.5))
    xi0 = min(x1,y1)
    xi1 = max(x2,y2)
    SDDGraphics.drawlinesegment(xi0,xi0,xi1,xi1)

    # Graphical analysis of the orbit of x0
    SDDGraphics.color(RGB(1,0,0))
    xi0 = x0
    for i in 1:iterations
        yi0 = f(xi0)
        SDDGraphics.drawlinesegment(xi0,xi0,xi0,yi0)
        SDDGraphics.drawlinesegment(xi0,yi0,yi0,yi0)
        xi0 = yi0
    end

    SDDGraphics.drawing()
end


#=

Lower bound instance from

http://proceedings.mlr.press/v28/jaggi13.pdf

and

https://arxiv.org/abs/1309.5550

Example instance is to minimize || x ||^2 over the probability simplex conv(e_1, ..., e_n)

Then the primal gap is lower bounded by 1/k - 1/n in iteration k as the optimal solution has value 1/n attained by
the (1/n, ..., 1/n) vector and in iteration k we have picked up at most k vertices from the simplex lower bounding the
primal value by 1/k.

Here: slightly rewritten to consider || x - (1/n, ..., 1/n) ||^2 so that the function value becomes directly the
primal gap (squared)

Three runs are compared:
1. Frank-Wolfe with traditional step-size rule
2. Away-step Frank-Wolfe with adaptive step-size rule
3. Blended Conditional Gradients with adaptive step-size rule

NOTE:
1. ignore the timing graphs
2. as the primal gap lower bounds the dual gap we also plot the primal gap lower bound in the dual gap graph
3. AFW violates the bound in the very first round which is due to the initialization in AFW starting with two vertices
4. all methods that call an LMO at most once in an iteration are subject to this lower bound
5. the objective is strongly convex, this implies limitations also for strongly convex functions (see for a discussion https://arxiv.org/abs/1906.07867)

=#

include("activate.jl")

import LinearAlgebra

# n = Int(1e1)
n = Int(1e2)
k = Int(1e3)

xp = 1 / n * ones(n);

# definition of objective
f(x) = LinearAlgebra.norm(x - xp)^2

# definition of gradient
function grad!(storage, x)
    @. storage = 2 * (x - xp)
end

# define LMO and do initial call to obtain starting point
lmo = FrankWolfe.ProbabilitySimplexOracle(1)
x0 = FrankWolfe.compute_extreme_point(lmo, zeros(n));

# simple benchmarking of oracles to get an idea how expensive each component is
FrankWolfe.benchmark_oracles(f, grad!, () -> rand(n), lmo; k=100)

@time x, v, primal, dual_gap, trajectory = FrankWolfe.frank_wolfe(
    f,
    grad!,
    lmo,
    x0,
    max_iteration=k,
    line_search=FrankWolfe.Agnostic(),
    L=2,
    print_iter=k / 10,
    emphasis=FrankWolfe.memory,
    verbose=true,
    epsilon=1e-5,
    trajectory=true,
);

@time x, v, primal, dual_gap, trajectoryAFW = FrankWolfe.away_frank_wolfe(
    f,
    grad!,
    lmo,
    x0,
    max_iteration=k,
    line_search=FrankWolfe.Adaptive(),
    print_iter=k / 10,
    emphasis=FrankWolfe.memory,
    verbose=true,
    epsilon=1e-5,
    trajectory=true,
);

@time x, v, primal, dual_gap, trajectoryBCG = FrankWolfe.blended_conditional_gradient(
    f,
    grad!,
    lmo,
    x0,
    max_iteration=k,
    line_search=FrankWolfe.Adaptive(),
    print_iter=k / 10,
    emphasis=FrankWolfe.memory,
    verbose=true,
    epsilon=1e-5,
    trajectory=true,
);

# define lower bound

trajLowerbound = []
for i in 1:n-1
    push!(trajLowerbound, (i, 1 / i - 1 / n, NaN, 1 / i - 1 / n, NaN))
end

data = [trajectory, trajectoryAFW, trajectoryBCG, trajLowerbound]
label = ["FW", "AFW", "BCG", "Lowerbound"]

# ignore the timing plots - they are not relevant for this example
FrankWolfe.plot_trajectories(data, label, xscalelog=true, legend_position=:bottomleft)


# Watts.jl

###### watts #####
"""
    watts(v, k)

    Compute the Watts Poverty Index of a vector `v` at a specified absolute
    poverty line `k`.

# Examples
```julia
julia> using Inequality
julia> watts([8, 5, 1, 3, 5, 6, 7, 6, 3], 4)
0.217962056224828
```
"""
watts(v::AbstractVector{<:Real}, k::Real)::Float64 = length(v[v .< k]) == 0 ?  0. : sum(log.(k./v[v .< k]))/length(v)

###### weighted watts #####
"""
    watts(v, w, α)

Compute the Watts Poverty Index of a vector `v` at a specified absolute
poverty line `α`, using weights given by a weight vector `w`.

Weights must not be negative, missing or NaN. The weights and data vectors must have the same length.

# Examples
```julia
julia> using Inequality
julia> watts([8, 5, 1, 3, 5, 6, 7, 6, 3], collect(0.1:0.1:0.9), 4)
0.17552777833850716
```
"""
function watts(v::AbstractVector{<:Real}, w::AbstractVector{<:Real}, α::Real)::Float64

    checks_weights(v, w)

    length(v) == 0 ? (return 0) : nothing

    return sum( (log.(α./(v[v .< α]))) .* w[v .< α] ) / sum(w)

end

function watts(v::AbstractVector{<:Real}, w::AbstractWeights, α::Real)::Float64

    checks_weights(v, w)

    length(v) == 0 ? (return 0) : nothing

    return sum( (log.(α./(v[v .< α]))) .* w[v .< α] ) / sum(w)

end

"""
    wwatts(v, w, α)

Compute the Watts Poverty Index of `v` with weights `w`. See also [`watts`](@watts)
"""
wwatts(v::AbstractVector{<:Real}, w::AbstractVector{<:Real}, α::Real)::Float64 = watts(v, w, α)

wwatts(v::AbstractVector{<:Real}, w::AbstractWeights, α::Real)::Float64 = watts(v, w, α)


# Perfect power function

@doc raw"""
```julia
isperfectpower(n::Integer) -> Bool
```

Given an integer `n`, returns `true` or `false` depending on whether or not it is a perfect power.

Here, a perfect power is some number of the form ``a^b``, where ``a, b \in \mathbb{N}``, and ``b > 1``.

This function is a wrapper around [Hecke.jl's really excellent and efficient `ispower` function](https://github.com/thofma/Hecke.jl/blob/master/src/Misc/Integer.jl#L413-L443).

!!! note

	It should be noted that there is another defintion for "perfect powers":
		> some number of the form ``p^b``, where ``p, b \in \mathbb{N}``, *and ``p`` is a prime number``*.
	Here, we call numbers of this form *prime powers*, or *proper perfect powers*.  By this definition, 36 is *not* a perfect power, because 6 is not prime.

See also: `CodingTheory.isperfectpower`.
"""
function isperfectpower(n::Integer)
    (n == 0) && return false
    e = first(ispower(n))
    return (e != 1 && e != 0)
end

@doc raw"""
```julia
isprimepower(n::Integer) -> Bool
```

Given an integer `n`, returns `true` or `false` depending on whether or not it is a prime power.

A prime power is some number of the form ``p^b``, where ``p, b \in \mathbb{N}``, and ``p`` is a prime number``.

This function is a wrapper around [Hecke.jl's really excellent and effcient `ispower` function](https://github.com/thofma/Hecke.jl/blob/master/src/Misc/Integer.jl#L756-L769).

See also: `CodingTheory.isperfectpower`.
"""
function isprimepower(n::Integer)
    return isprime_power(n)
end

## Prime power function

#=
function __isprime(n)
	return isprime(fmpz(n))
end

function __factors(n)
	return n == 0 ? [] : factor(fmpz(n))
end
# function __factors(n)
# 	first(i) for i in Primes.factor(n)
# end

function primedivisors(n)
    n == 0 && return []
    __isprime(n) && return [fmpz(n)]
    f = __factors(n)
    return sort!([p for (p, e) ∈ f])
end

function ω(n)
	nprimedivisors = 0
	if n == 0
		nprimedivisors = 0
	elseif __isprime(n)
		nprimedivisors = 1
    else
        nprimedivisors = length(__factors(n))
	end

	return fmpz(nprimedivisors)

    # # original definition:
	# return fmpz(length(primedivisors(n)))
end

function Ω(n)
    n == fmpz(0) && return 0
    __isprime(n) && return fmpz(1)

    return sum(e for (__, e) ∈ __factors(n))
end

# this is a slow version compared to the Hecke one
function isperfectpower(n)
	return ω(n) == 1 && Ω(n) != 1
end

# this is slower than the Hecke verson
function isprimepower(n)
    return ω(n) == 1
end

=#


"""
    logistic_key(x_init, r, num_keys; scaling_factor=10.0^16, upper_bound=256.0)

Generates a vector of pseudo-random keys using the Logistic Map.

The equation -

\$x_{n+1} = r * x_{n} * (1 - x_{n})\$

# Arguments
- `x_init::Float64`: Initial value of x. x ϵ (0, 1).
- `r::Float64`: A constant value. Values > 4 usually results in pseudo-random numbers.
- `num_keys::Int64`: Number of keys to be generated.
- `scaling_factor::Float64=10.0^16`: Factor to be multiplied to the generated value of pseudo-random
    number. Ideally, the factor should be > upper_bound.
- `upper_bound::Float64=256.0`: Upper bound of keys (not included). Use 256 for encrypting images
    as the RGB values of a pixel varies from 0 to 255.

# Returns
- `keys::Vector{Int64}:`: Generated pseudo-random keys.

# Example
```jldoctest
julia> logistic_key(0.01, 3.97, 20)
20-element Vector{Int64}:
   0
  44
   7
  26
  14
 224
  16
 250
 162
 211
 200
 217
  97
 132
 134
 100
 135
 232
 122
 102
```
"""
function logistic_key(
    x_init::Float64,
    r::Float64,
    num_keys::Int64;
    scaling_factor::Float64=10.0^16,
    upper_bound::Float64=256.0
)
    keys = Vector{Int64}(undef, num_keys)
    x = x_init

    for i = 1:num_keys
        x = r * x * (1 - x)
        key = x * scaling_factor % upper_bound
        keys[i] = trunc(Int, key)
    end

    return keys
end


using Random
using PyPlot
using Printf

function value(x_true,y_true,x_sensor,y_sensor)
    return 1/(0.1+ (x_true-x_sensor)^2 + (y_true-y_sensor)^2)
end

function readearthquakedata(filename)
    lines = open(filename) do file
        readlines(file)
    end
    [parse(Float64, i) for i in lines]
end

function spiralcoordinates(S, rate)
    x=zeros(S); y=zeros(S)
    for s=1:S
        theta=rate*2*pi*s/S;  r=s/S
        x[s]=r*cos(theta); y[s]=r*sin(theta)
    end
    return x, y
end

function sensorcoordinates(N)
    x_sensor=zeros(N); y_sensor=zeros(N)
    for sensor=1:N
        theta_sensor=2*pi*sensor/N
        x_sensor[sensor]=cos(theta_sensor); y_sensor[sensor]=sin(theta_sensor)
    end
    return x_sensor, y_sensor
end

function sensorexplosionvalues(S, N, x, y, x_sensor, y_sensor)
    v=zeros(S,N)
    for sensor=1:N
        for s=1:S
            v[s,sensor]=value(x[s],y[s],x_sensor[sensor],y_sensor[sensor]) # explosion value for some value function
        end
    end
    return v
end

function logprobabilities_twoexplosions(S, N, sd, val, v)
    logp = zeros(S,S)
    for s1=1:S
        for s2=1:S
            for sensor=1:N
                logp[s1,s2] += -0.5*(val[sensor]-v[s1,sensor]-v[s2,sensor])^2/(sd^2)
            end
        end
    end
    return logp
end

function logprobabilities_oneexplosion(S, N, sd, val, v)
    logp = zeros(S)
    for s=1:S
        for sensor=1:N
            logp[s] += -0.5*(val[sensor]-v[s,sensor])^2/(sd^2)
        end
    end
    return logp
end

function plotboundary()
    for theta=0:0.01:2*pi
        plot(cos(theta),sin(theta),".",color=[0,0,0])
    end
end

function plotsensors(x_sensor, y_sensor)
    for sensor=1:length(x_sensor)
        plot(x_sensor[sensor],y_sensor[sensor],"o",color=[1,0,0])
    end
end

function plotprobabilities(p, x, y)
    maxp, maxpind = findmax(p)
    for s=1:length(x)
        plot(x[s],y[s],".",color=(1-(p[s]/maxp))*[1,1,1])
    end
end

function plotposterior(filename, valuescale)
    S = 2000
    rate = 25
    sd = 0.2

    x, y = spiralcoordinates(S, rate)

    N=30 # number of stations

    x_sensor, y_sensor = sensorcoordinates(N)

    v = valuescale.*sensorexplosionvalues(S, N, x, y, x_sensor, y_sensor)

    val = readearthquakedata(filename)

    logp = logprobabilities_twoexplosions(S, N, sd, val, v)
    p=exp.(logp.-maximum(logp))
    p=p/sum(p)

    maxp, maxind = findmax(p)
    # Plot most likely locations
    figure()
    p1 = sum(p, dims=2)
    plotprobabilities(p1, x, y)
    plotboundary()
    plotsensors(x_sensor, y_sensor)
    plot(x[maxind[1]],y[maxind[1]],"m+",markersize=20,label="explosion 1")
    plot(x[maxind[2]],y[maxind[2]],"m+",markersize=20,label="explosion 2")

end

function plotposterioroneexplosion(filename, valuescale)
    S = 2000
    rate = 25
    sd = 0.2

    x, y = spiralcoordinates(S, rate)

    N=30 # number of stations

    x_sensor, y_sensor = sensorcoordinates(N)

    v = valuescale.*sensorexplosionvalues(S, N, x, y, x_sensor, y_sensor)

    val = readearthquakedata(filename)

    logp = logprobabilities_oneexplosion(S, N, sd, val, v)
    p=exp.(logp.-maximum(logp))
    p=p/sum(p)

    maxp, maxind = findmax(p)
    # Plot most likely locations
    figure()
    plotprobabilities(p, x, y)
    plotboundary()
    plotsensors(x_sensor, y_sensor)
    plot(x[maxind],y[maxind],"m+",markersize=20,label="explosion")
end

function hypothesislogratio(filename, valuescale)
    S = 2000
    rate = 25
    sd = 0.2

    x, y = spiralcoordinates(S, rate)

    N=30 # number of stations

    x_sensor, y_sensor = sensorcoordinates(N)

    v = valuescale.*sensorexplosionvalues(S, N, x, y, x_sensor, y_sensor)

    val = readearthquakedata(filename)

    logp2 = logprobabilities_twoexplosions(S, N, sd, val, v)
    maxp2 = maximum(logp2)
    p2=exp.(logp2.-maxp2)
    lp2 = log(sum(p2))+maxp2

    logp1 = logprobabilities_oneexplosion(S, N, sd, val, v)
    maxp1 = maximum(logp1)
    p1 = exp.(logp1.-maxp1)
    lp1 = log(sum(p1))+maxp1

    println(lp2)
    println(lp1)
    return lp2-lp1
end

function assignment1()
    firstdatafn = "EarthquakeExerciseData.txt"
    plotposterior(firstdatafn, 1.0)

    meandatafn = "EarthquakeExerciseMeanData.txt"
    plotposterior(meandatafn, 0.5)

    plotposterioroneexplosion(firstdatafn, 1.0)

    ratio = hypothesislogratio(firstdatafn, 1.0)
    println("Hypothesis log ratio for first data:")
    println("$(@sprintf("%.2f", ratio))")

    ratio2 = hypothesislogratio(meandatafn, 0.5)
    println("Hypothesis log ratio for mean data:")
    println("$(@sprintf("%.2f", ratio2))")
end

Earthqauke - Assignment 1/.ipynb_checkpoints/assignment1-checkpoint.jl


# // process synthes1
# // mixed-integer nonlinear program
#
# // Source: Test problem 1 (Synthesis of processing system) in
# // M. Duran & I.E. Grossmann,
# // "An outer approximation algorithm for a class of mixed integer nonlinear
# //  programs", Mathematical Programming 36, pp. 307-339, 1986.
#
# // Number of variables:   6 (3 binary variables)
# // Number of constraints: 6
# // Objective nonlinear
# // Nonlinear constraints

function minlp_demo(gm::Bool = false)
    m = JuMP.Model()
    @variable(m, x[1:6] >= 0)
    JuMP.set_binary.(x[4:6])
    JuMP.set_upper_bound(x[1], 2)
    JuMP.set_upper_bound(x[2], 2)
    JuMP.set_upper_bound(x[3], 1)
    JuMP.set_upper_bound(x[4], 1)
    JuMP.set_upper_bound(x[5], 1)
    JuMP.set_upper_bound(x[6], 1)

    @objective(m, Min, 10x[1] - 17x[3] -5x[4] + 6x[5] + 8x[6])

    @constraint(m, c3, x[2] - x[1] <= 0)
    @constraint(m, c4, x[2] - 2*x[4] <= 0)
    @constraint(m, c5, x[1] - x[2] - 2*x[5] <= 0)
    @constraint(m, c6, x[4] + x[5] <= 1)

    if !gm
        @NLconstraint(m, c1, 0.8*log(x[2] + 1) + 0.96*log(x[1] - x[2] + 1) - 0.8*x[3] >= 0)
        @NLconstraint(m, c2, log(x[2] + 1) + 1.2*log(x[1] - x[2] + 1) - x[3] - 2*x[6] >= -2)
        set_optimizer(m, BARON_SILENT)
        return m
    else
        gm = GlobalModel(model = m, name = "minlp")
        set_optimizer(gm, CPLEX_SILENT)
        add_nonlinear_constraint(gm, :(x -> 0.8*log(x[2] + 1) + 0.96*log(x[1] - x[2] + 1) - 0.8*x[3]),
                                vars =[x[1], x[2], x[3]], name = "c1")
        add_nonlinear_constraint(gm, :(x -> log(x[2] + 1) + 1.2*log(x[1] - x[2] + 1) - x[3] - 2*x[6] + 2),
                                vars =[x[1], x[2], x[3], x[6]], name = "c2")
        return gm
    end
end


export isprime, nextprime, gen_n_primes, brute_next_factor, next_factor

function isprime(x::Int64)::Bool
        if x % 2 == 0; return false; end
        for i in 3:2:Int(ceil(sqrt(x)))
                if x % i == 0; return false; end
        end
        return true
end

function nextprime(lastprime::Int64)
        if lastprime == 2; return 3; end
        lastprime += lastprime % 2 == 0 ? 1 : 2
        while !isprime(lastprime); lastprime += 2; end
        return lastprime
end

function gen_n_primes(n::Int64)
        out = zeros(Int64, n)
        idx = 2
        out[1] = 2
        while idx < n + 1
                out[idx] = nextprime(out[idx - 1])
                idx += 1
        end
        return out
end

function brute_next_factor(x::T)::T where T
        if x % 2 == 0
                return x == 2 ? 1 : 2
        elseif x % 3 == 0
                return x == 3 ? 1 : 3
        elseif x % 5 == 0
                return x == 5 ? 1 : 5
        end
        for i in 3:2:T(ceil(sqrt(x)))
                if i % 5 != 0 && x % i == 0; return T(i); end
        end
        return 1
end

"""Factoring Algorithm made by Johnathan Bizzano
   W is the diagnol offset such that (maxsqr + w)(maxsqr - w) = x
"""

function next_factor(x::T) where T <: Real
        if x % 2 == 0
                return x == 2 ? 1 : 2
        elseif x % 3 == 0
                return x == 3 ? 1 : 3
        elseif x % 5 == 0
                return x == 5 ? 1 : 5
        end
        last_factor = 7
        next_factor = 11
        last_func = x - last_factor * floor(x / last_factor)

        for i in 1:100
                func = x - next_factor * floor(x / next_factor)
                if func == 0 || isnan(next_factor)
                        return isnan(next_factor) ? 1 : next_factor
                end
                deriv = (func - last_func) / (next_factor - last_factor)
                last_func = func
                last_factor = next_factor
                next_factor -= func / (deriv)
        end

        return isnan(next_factor) ? 1 : (next_factor < 1 ? 1 : next_factor)
end

function next_factor(x::T) where T <: Complex
        if cplx_mod(x, 2) == 0
                return x == 2 ? 1 : 2
        elseif cplx_mod(x, 3) == 0
                return x == 3 ? 1 : 3
        elseif cplx_mod(x, 5) == 0
                return x == 5 ? 1 : 5
        end
        next_sqr_factor = cplx_ceil(sqrt(x))
        dist = next_sqr_factor * next_sqr_factor - x
        factor = next_sqr_factor - sqrt(next_sqr_factor * next_sqr_factor - x)
        if factor == cplx_floor(factor); return factor; end
        return 1
end


#= Finding the Nth number in the Fibonacci Sequence using Dynamic Programming
=#

## Function

function fibonacci(n)
    f = Int64[]
    push!(f, 0)
    push!(f, 1)
    for i = 3:n
        temp = f[i-1] + f[i-2]
        push!(f, temp)
    end
    return f[n]
end

## Input

print("Enter the number: ")
n = readline()
n = parse(Int64, n)
println("The $(n)th fibonacci number is ")

## Calling the function

fibonacci(n)

#=
Sample Test Case:
Input:
    Enter the number: 5
Output:
    The 5th fibonacci number is 3

Time Complexity: O ( N )
=#


"""
    u(t::Float64)::Float64

Unit Step Function

``\\mathrm{u}(t)= \\begin{cases} 1, &   t \\geq 0 \\\\ 0, &   t < 0 \\end{cases}``

# Examples
```@example
using ISA
t = 2.5
u(t)
```
"""
function u(t::Float64)::Float64 #unit step function
    return ifelse( t < 0, zero(t), one(t) )
end

"""
    𝒩ᵤ(x::Float64; μ::Float64, σ::Float64)::Float64

Unnormalized Gaussian Function

``\\mathcal{N}_\\mathrm{u}(x;\\mu,\\sigma) = \\exp\\left(-\\frac{1}{2}\\left(\\frac{x-\\mu}{\\sigma}\\right)^2\\right).``

# Examples
```@example
using ISA
x = 2.0
μ = 1.0
σ = 0.1
𝒩ᵤ(x; μ, σ)
```
"""
function 𝒩ᵤ(x::Float64; μ::Float64, σ::Float64)::Float64
    return exp(-0.5 * ((x - μ) / σ)^2)
end

"""
    𝒩(x::Float64; μ::Float64, σ::Float64)::Float64

Unit Area Gaussian Function

``\\mathcal{N}(x;\\mu,\\sigma) = \\frac{1}{\\sqrt{2\\pi}\\sigma}\\exp\\left(-\\frac{1}{2}\\left(\\frac{x-\\mu}{\\sigma}\\right)^2\\right).``

# Examples
```@example
using ISA
x = 2.0
μ = 1.0
σ = 0.1
𝒩(x; μ, σ)
```
"""
function 𝒩(x::Float64; μ::Float64, σ::Float64)::Float64
    return 1 / (σ * sqrt(2 * π)) * exp(-0.5 * ((x - μ) / σ)^2)
end

"""
    δn(t::Float64,σ::Float64=1.0e-11)::Float64

Normalized Dirac Delta Approximation

``δ_n(t;\\sigma) = \\exp\\left(-\\frac{1}{2}\\left(\\frac{t^2}{\\sigma^2}\\right)\\right). ``

# Examples
```@example
using ISA
t = 1.0e-10
σ = 1.0e-11
δn(t; σ)
```
"""
function δn(t::Float64,σ::Float64=1.0e-11)::Float64
   return 𝒩ᵤ(t, μ=0.0, σ=σ)
end

"""
    δ(t::Float64,σ::Float64=1.0e-11)::Float64

Dirac Delta Approximation

``δ(t;\\sigma) = \\frac{1}{\\sqrt{2\\pi}\\sigma}\\exp\\left(-\\frac{1}{2}\\left(\\frac{t^2}{\\sigma^2}\\right)\\right). ``

# Examples
```@example
using ISA
t = 1.0e-10
σ = 1.0e-11
δ(t; σ)
```
"""
function δ(t::Float64,σ::Float64=1.0e-11)::Float64
   return 𝒩(t, μ=0.0, σ=σ)
end


# This file is a part of AstroLib.jl. License is MIT "Expat".
# Copyright (C) 2016 Mosè Giordano.

function _aitoff{T<:AbstractFloat}(l::T, b::T)
    l > 180.0 && (l -= 360.0)
    alpha2 = deg2rad(l/2.0)
    delta = deg2rad(b)
    r2 = sqrt(2.0)
    f = 2.0*r2/pi
    cdec = cos(delta)
    denom = sqrt(1.0 + cdec*cos(alpha2))
    return rad2deg(cdec*sin(alpha2)*2.0*r2/denom/f), rad2deg(sin(delta)*r2/denom/f)
end

"""
    aitoff(l, b) -> x, y

### Purpose ###

Convert longitude `l` and latitude `b` to `(x, y)` using an Aitoff projection.

### Explanation ###

This function can be used to create an all-sky map in Galactic coordinates with
an equal-area Aitoff projection.  Output map coordinates are zero longitude
centered.

### Arguments ###

* `l`: longitude, scalar or vector, in degrees.
* `b`: latitude, number of elements as `l`, in degrees.

Coordinates can be given also as a 2-tuple `(l, b)`.

### Output ###

2-tuple `(x, y)`.

* `x`: x coordinate, same number of elements as `l`.  `x` is normalized to be in
  \$[-180, 180]\$.
* `y`: y coordinate, same number of elements as `l`.  `y` is normalized to be in
  \$[-90, 90]\$.

### Example ###

Get \$(x ,y)\$ Aitoff coordinates of Sirius, whose Galactic coordinates are
\$(227.23, -8.890)\$.

``` julia
x, y = aitoff(227.23, -8.890)
# => (-137.92196683723276,-11.772527357473054)
```

### Notes ###

See AIPS memo No. 46
(ftp://ftp.aoc.nrao.edu/pub/software/aips/TEXT/PUBL/AIPSMEMO46.PS), page 4, for
details of the algorithm.  This version of `aitoff` assumes the projection is
centered at b=0 degrees.

Code of this function is based on IDL Astronomy User's Library.
"""
aitoff(l::Real, b::Real) = _aitoff(promote(float(l), float(b))...)

aitoff(lb::Tuple{Real, Real}) = aitoff(lb...)

function aitoff{L<:Real,B<:Real}(l::AbstractArray{L}, b::AbstractArray{B})
    @assert length(l) == length(b)
    typel = typeof(float(one(L)))
    x = similar(l, typel)
    y = similar(b, typel)
    for i in eachindex(l)
        x[i], y[i] = aitoff(l[i], b[i])
    end
    return x, y
end


function convolve_beam(NN::Int, pix_size::Float64, beam_waist::Float64,
                       Map::Array{Float64, 2})

    gaussian = make_2d_gaus(NN, pix_size, beam_waist)

    FT_beam = fft(fftshift(gaussian))
    FT_Map  = fft(fftshift(Map))
    conv = FT_beam .* FT_Map

    conv_map = fftshift(real.(ifft(conv)))

    return conv_map

end

function make_2d_gaus(N::Int, pix_size::Float64, beam_waist::Float64)

    inds = ((0:(N-1)) .+ 0.5 .- (N/2)) * pix_size
    X = reshape(repeat(inds, N), N ,N)
    Y = X'
    R = (X .^2 .+ Y .^2) .^0.5

    sigma = beam_waist / sqrt(8. * log(2))
    gauss = exp.(-0.5 .* ((R ./ sigma).^2.0))
    Norm_Fact = sum(gauss)
    gauss_norm = gauss ./ Norm_Fact

    return gauss_norm

end


# Strange attractors from https://www.dynamicmath.xyz/strange-attractors/ and
# https://en.wikipedia.org/wiki/List_of_chaotic_maps
# Opted for the equations as reported in papers

# Thomas
@parameters t b=0.208186
@variables x(t)=1 y(t)=0 z(t)=0
D = Differential(t)

eqs = [D(x) ~ sin(y) - b*x,
       D(y) ~ sin(z) - b*y,
       D(z) ~ sin(x) - b*z]

@named thomas = ODESystem(eqs)

"""
Thomas' cyclically symmetric attractor equations

```math
$(latexify(thomas))
```

[Reference](https://www.worldscientific.com/doi/abs/10.1142/S0218127499001383)

[Wikipedia](https://en.wikipedia.org/wiki/Thomas%27_cyclically_symmetric_attractor)
"""
prob_ode_thomas= ODEProblem(thomas,[],(0.0,1.0))

# Lorenz
@parameters t σ=10 ρ=28 β=8/3
@variables x(t)=1 y(t)=0 z(t)=0
D = Differential(t)

eqs = [D(x) ~ σ*(y-x),
       D(y) ~ x*(ρ-z)-y,
       D(z) ~ x*y - β*z]

@named lorenz = ODESystem(eqs)

"""
Lorenz equations

```math
$(latexify(lorenz))
```

[Reference](https://journals.ametsoc.org/view/journals/atsc/20/2/1520-0469_1963_020_0130_dnf_2_0_co_2.xml)

[Wikipedia](https://en.wikipedia.org/wiki/Lorenz_system)
"""
prob_ode_lorenz = ODEProblem(lorenz,[],(0.0,1.0))

# Aizawa
@parameters t a=0.95 b=0.7 c=0.6 d=3.5 e=0.25 f=0.1
@variables x(t)=1 y(t)=0 z(t)=0
D = Differential(t)

eqs = [D(x) ~ (z-b)*x-d*y,
       D(y) ~ d*x+(z-b)*y,
       D(z) ~ c+a*z-z^3/3-(x^2+y^2)*(1+e*z)+ f*z*x^3]

@named aizawa = ODESystem(eqs)

"""
Aizawa equations

```math
$(latexify(aizawa))
```

[Reference](https://journals.ametsoc.org/view/journals/atsc/20/2/1520-0469_1963_020_0130_dnf_2_0_co_2.xml)

"""
prob_ode_aizawa = ODEProblem(aizawa,[],(0.0,1.0))

# Dadras
@parameters t a=3 b=2.7 c=1.7 d=2 e=9
@variables x(t)=1 y(t)=0 z(t)=0
D = Differential(t)

eqs = [D(x) ~ y-a*x+b*y*z,
       D(y) ~ c*y-x*z+z,
       D(z) ~ d*x*y-e*z]

@named dadras = ODESystem(eqs)

"""
Dadras equations

```math
$(latexify(dadras))
```

[Reference](https://www.sciencedirect.com/science/article/abs/pii/S0375960109009591)

"""
prob_ode_dadras = ODEProblem(dadras,[],(0.0,1.0))

# chen
@parameters t a=35 b=3 c=28
@variables x(t)=1 y(t)=0 z(t)=0
D = Differential(t)

eqs = [D(x) ~ a*(y-x),
       D(y) ~ (c-a)*x-x*z+c*y,
       D(z) ~ x*y-b*z]

@named chen = ODESystem(eqs)

"""
chen equations

```math
$(latexify(chen))
```

[Reference](https://www.worldscientific.com/doi/abs/10.1142/S0218127499001024)

"""
prob_ode_chen = ODEProblem(chen,[],(0.0,1.0))

# rossler
@parameters t a=0.2 b=0.2 c=5.7
@variables x(t)=1 y(t)=0 z(t)=0
D = Differential(t)

eqs = [D(x) ~ -(y+z),
       D(y) ~ x+a*y,
       D(z) ~ b+z*(x-c)]

@named rossler = ODESystem(eqs)

"""
rossler equations

```math
$(latexify(rossler))
```

[Reference](https://www.sciencedirect.com/science/article/abs/pii/0375960176901018)
[Wikipedia](https://en.wikipedia.org/wiki/R%C3%B6ssler_attractor)

"""
prob_ode_rossler = ODEProblem(rossler,[],(0.0,1.0))

# rabinovich_fabrikant
@parameters t a=0.14 b=0.10
@variables x(t)=1 y(t)=0 z(t)=0
D = Differential(t)

eqs = [D(x) ~ y*(z-1+x^2) + b*x,
       D(y) ~ x*(3*z+1-x^2)+b*y,
       D(z) ~ -2*z*(a+x*y)]

@named rabinovich_fabrikant = ODESystem(eqs)

"""
rabinovich_fabrikant equations

```math
$(latexify(rabinovich_fabrikant))
```

[Reference](https://en.wikipedia.org/wiki/Rabinovich%E2%80%93Fabrikant_equations)

"""
prob_ode_rabinovich_fabrikant = ODEProblem(rabinovich_fabrikant,[],(0.0,1.0))

# sprott
@parameters t a=2.07 b=1.79
@variables x(t)=1 y(t)=0 z(t)=0
D = Differential(t)

eqs = [D(x) ~ y+a*x*y+x*z,
       D(y) ~ 1-b*x^2+y*z,
       D(z) ~ x-x^2-y^2]

@named sprott = ODESystem(eqs)

"""
sprott equations

```math
$(latexify(sprott))
```

[Reference](http://sprott.physics.wisc.edu/pubs/paper423.pdf)

"""
prob_ode_sprott = ODEProblem(sprott,[],(0.0,1.0))

# hindmarsh_rose
@parameters t a=1 b=3 c=1 d=5 r=1e-2 s=4 xr=-8/5 i=5
@variables x(t)=1 y(t)=0 z(t)=0
D = Differential(t)

eqs = [D(x) ~ y-a*x^3+b*x^2-z+i,
       D(y) ~ c-d*x^2-y,
       D(z) ~ r*(s*(x-xr)-z)]

@named hindmarsh_rose = ODESystem(eqs)

"""
hindmarsh_rose equations

```math
$(latexify(hindmarsh_rose))
```

[Reference](https://en.wikipedia.org/wiki/Hindmarsh%E2%80%93Rose_model)

"""
prob_ode_hindmarsh_rose = ODEProblem(hindmarsh_rose,[],(0.0,1.0))


"""
    LogSpaced(min::Float64, max::Float64, n::Int64)

This function evaluates ``n`` points, logarithmically spaced between
    ``min`` and ``max``.
"""
function LogSpaced(min::Float64, max::Float64, n::Int64)
    logmin = log10(min)
    logmax = log10(max)
    logarray = Array(LinRange(logmin, logmax, n))
    return exp10.(logarray)
end

"""
    ΔLogSpaced(min::Float64, max::Float64, n::Int64)

This function evaluates ``n`` points, logarithmically spaced between
    ``min`` and ``max``.
"""
function ΔLogSpaced(min::Float64, max::Float64, n::Int64)
    λmin = log10(min)
    λmax = log10(max)
    Δℓ = zeros(n)
    for i in 1:n
        Δℓ[i] = 10^(λmin+i*(λmax-λmin)/n)-10^(λmin+(i-1)*(λmax-λmin)/n)
    end
    return Δℓ
end

"""
    BinSearch(x::Float64, Array::Vector{Float64})

Given a value ``z`` and an Array, determines the couple of array elements where
``z`` lies and returns the index corresponding to the first value.
"""
function BinSearch(x::Float64, Array::Vector{Float64})
    idx = 1
    if x <= first(Array)
        idx = 1
    elseif x >= last(Array)
        idx = length(Array)-1
    else
        while !(x >= Array[idx] && x <= Array[idx+1])
            idx += 1
        end
    end
    return idx
end

"""
    CustomRegression(x::Vector{Float64}, y::Vector{Float64})

Given two arrays ``x`` and ``y``, performs the linear regression and returns the
coefficients ``c`` and ``m`` of the fitted line.
"""
function CustomRegression(x::Vector{Float64}, y::Vector{Float64})
    x_mean = mean(x)
    y_mean = mean(y)
    m = 0
    den = 0
    for i in 1:length(x)
        m += (x[i]-x_mean)*(y[i]-y_mean)
        den  += (x[i]-x_mean)^2
    end
    m /= den
    c = y_mean - m* x_mean
    return c, m
end

"""
    SimpsonWeightArray(n::Int64)

This function evaluates an array with the Simpson weight, for Simpson
Integration
"""
function SimpsonWeightArray(n::Int64)
    number_intervals = floor((n-1)/2)
    weight_array = zeros(n)
    if n == number_intervals*2+1
        for i in 1:number_intervals
            weight_array[Int((i-1)*2+1)] += 1/3
            weight_array[Int((i-1)*2+2)] += 4/3
            weight_array[Int((i-1)*2+3)] += 1/3
        end
    else
        weight_array[1] += 0.5
        weight_array[2] += 0.5
        for i in 1:number_intervals
            weight_array[Int((i-1)*2+1)+1] += 1/3
            weight_array[Int((i-1)*2+2)+1] += 4/3
            weight_array[Int((i-1)*2+3)+1] += 1/3
        end
        weight_array[length(weight_array)]   += 0.5
        weight_array[length(weight_array)-1] += 0.5
        for i in 1:number_intervals
            weight_array[Int((i-1)*2+1)] += 1/3
            weight_array[Int((i-1)*2+2)] += 4/3
            weight_array[Int((i-1)*2+3)] += 1/3
        end
        weight_array ./= 2
    end
    return weight_array
end

"""
    SimpsonWeightMatrix(n::Int64)

This function evaluates an array with the Simpson weight, for Simpson
Integration of the Lensing Efficiency
"""
function SimpsonWeightMatrix(n::Int64)
    number_intervals = floor((n-1)/2)
    weight_matrix = zeros(n, n)
    for i in 1:n
        weight_matrix[i,i:n] = SimpsonWeightArray(n-i+1)
    end
    return weight_matrix
end

"""
    Difference(InputArray::Vector{Float64})

This function evaluates the n-th discrete difference of a given 1-D array.
"""
function Difference(InputArray::Vector{Float64})
    OutputArray = zeros(length(InputArray)-1)
    for i in 1:length(InputArray)-1
        OutputArray[i] = InputArray[i+1] - InputArray[i]
    end
    return OutputArray
end

"""
    UnevenTrapzWeightArray(InputArray::Vector{Float64})

This function evaluates the array of weights for the integration with uneven
spaced points and the Trapezoidal rule.
"""
function UnevenTrapzWeightArray(InputArray::Vector{Float64})
    WeightArray = zeros(length(InputArray))
    DifferenceArray = Difference(InputArray)
    for idx in 2:length(WeightArray)-1
        WeightArray[idx] = (DifferenceArray[idx]+DifferenceArray[idx-1])/2
    end
    WeightArray[1] = DifferenceArray[1]/2
    WeightArray[length(InputArray)] = DifferenceArray[length(InputArray)-1]/2
    return WeightArray
end

function UnevenTrapzWeightMatrix(InputMatrix::AbstractArray{Float64, 2})
    WeightMatrix = zeros(size(InputMatrix))
    for lidx in 1:length(WeightMatrix[:,1])
        WeightMatrix[lidx,:] = UnevenTrapzWeightArray(InputMatrix[lidx,:])
    end
    return WeightMatrix
end

function uᵢⱼ(n::Int, i::Int, j::Int)
    u = zeros(floor(Int,0.5*n*(n+1)))
    u[floor(Int64, (j-1)*n+i-0.5*j*(j-1))] = 1
    return u
end

function eᵢ(n::Int, i::Int)
    e = zeros(n)
    e[i] = 1
    return e
end

function Eᵢⱼ(n::Int, i::Int, j::Int)
    E = zeros((n,n))
    E[i,j] = 1
    return E
end

function Tᵢⱼ(n::Int, i::Int, j::Int)
    if i != j
        T = Eᵢⱼ(n,i,j) + Eᵢⱼ(n,j,i)
    else
        T = Eᵢⱼ(n,i,i)
    end
    return T
end

function EliminationMatrix(n::Int)
    L = zeros((floor(Int64,0.5*n*(n+1)), n*n))
    for i in 1:n
        for j in 1:i
            L += kron(uᵢⱼ(n,i,j), transpose(vec(Eᵢⱼ(n,i,j))))
        end
    end
    return L
end

"""
    DuplicationMatrix(n::Int)

Duplication matrix ``\\boldsymbol{D}_n``.
"""
function DuplicationMatrix(n::Int)
    D = zeros((n*n, floor(Int64,0.5*n*(n+1))))
    for i in 1:n
        for j in 1:i
            D += transpose(kron(uᵢⱼ(n,i,j), transpose(vec(Tᵢⱼ(n,i,j)))))
        end
    end
    return D
end


"""Julia program to implement Jump Search algorithm.
Jump Search is a searching algorithm similar to binary search, where a sorted array is searched.
In this algorithm, only fewer comparisons are done, by jumping ahead some elements while traversal
"""

function jump_search(arr, num, ele)

    # Size by which the traversing index should be jumped is calculated.
    jump = floor(Int, sqrt(num))
    step = jump
    temp = 1

    """ The entire sorted array is divided into blocks of sqrt(n) size and the block just after which the
        required element is present is being identified. and the previous block starting index is stored in temp"""
    while (arr[min(step, num)] < ele)
        temp = step
        step = step + jump
        if (temp >= num)
            return false
        end
    end

    # The identified block, is searched via linea search for the required element.
    while (arr[temp] < ele)
        temp = temp + 1
        if (arr[temp] == ele)
            return true
        end
        # Check if the end of the array or end of the block is reached
        if (temp == min(step, num))
            return false
        end
    end
    if (arr[temp] == ele)
        return true
    else
        return false
    end
end

print("How many numbers are present in the array? ")
n = readline()
n = parse(Int, n)
if (n <= 0)
    println("Array is Empty!!!")
    exit()
end
arr = Int[]
print("Enter the numbers: ")
arr = [parse(Int, num) for num in split(readline())]
print("Which number do you want to search in the array? ")
ele = readline()
ele = parse(Int, ele)
# Sort the array in ascending order
arr = sort(arr)
res = jump_search(arr, n, ele)
if (res == 0)
    print("The number $ele is not present in the array")
else
    print("The number $ele is present in the array.")
end

"""
Time Complexity - O(n^(0.5)), where 'n' is the size of the array
Space Complexity - O(n)

SAMPLE INPUT AND OUTPUT

SAMPLE I

How many numbers are present in the array? 5
Enter the numbers: 1 2 3 4 5
Which number do you want to search in the array? 6
The number 6 is not present in the array

SAMPLE II

How many numbers are present in the array? 3
Enter the numbers: 3 1 2
Which number do you want to search in the array? 2
The number 2 is present in the array.

"""


"""Julia program to implement Painting Fence algorithm
Given a fence with n posts and k colors
Find out the number of ways of painting the fence
such that at most 2 adjacent posts have the same color.

"""

function painting_fence(n, k)
    if (k < (n ÷ 2) || n < 2)
        return -1
    end
    tot = [0 for i in 1:(n - 1)]
    same = [0 for i in 1:(n - 1)]
    diff = [0 for i in 1:(n - 1)]

    same[1] = k
    diff[1] = k * (k - 1)
    tot[1] = k * k
    for i in 2:(n-1)
        same[i] = diff[i-1]
        diff[i] = tot[i-1] * (k-1)
        tot[i] = same[i] + diff[i]
    end
    return tot[n-1]
end

print("Enter the number of posts: ")
n = readline()
n = parse(Int, n)
print("Enter the number of colours available: ")
k = readline()
k = parse(Int, k)
ans = painting_fence(n, k)
if( ans == -1)
    println("The given posts cannot be painted with the given colours satisfying the given conditions")
else
    println("The number of ways to paint the given posts is $ans")
end

"""
Time Complexity: O(n), where 'n' is the number of posts
Space Complexity: O(n)

SAMPLE INPUT AND OUTPUT

SAMPLE 1

Enter the number of posts: 10
Enter the number of colours available: 4
The given posts cannot be painted with the given colours satisfying the given conditions

SAMPLE 2

Enter the number of posts: 3
Enter the number of colours available: 5
The number of ways to paint the given posts is 120
"""


# Julia program to find the number of bits to be flipped to convert one number to another

"""Inorder to compare the individual bits of given the numbers, we can compute the number of
set bits in thier XOR, as we know that XOR operation yields set bits, only when two same
bits are XORed."""
function get_num_flips(n1, n2)
    count = 0
    n = n1 ⊻ n2
    while(n != 0)
        if(n & 1 == 1)
            count = count + 1
        end
        n = n >> 1
    end
    return count
end

print("Enter the first number:")
n1 = readline()
n1 = parse(Int32, n1)
print("Enter the second number:")
n2 = readline()
n2 = parse(Int32, n2)
res = get_num_flips(n1, n2)
print("Number of flips required to inter-convert the given numbers is: $res")

"""
Time Complexity: O(log(n)), where n is the greater of the input number
Space Complexity: O(1)

SAMPLE INPUT AND OUTPUT

Enter the first number:13
Enter the second number:26
Number of flips required to inter-convert the given numbers is: 4
"""


using Polynomials, Roots, JLD, PyPlot
using JFVM
include("peteng-master/functions/rel_perms_real.jl")

pce = 1e6
swc = 0.12
sor = 0.08

figure()
sw_plot = collect(linspace(0,1,1000))
plot(sw_plot, pc_drain.(sw_plot, pce, swc), sw_plot, pc_imb.(sw_plot, pce, swc, sor))
title("Capillary Pressure, "L"$p_c$"" [Pa]")
xlabel(L"$S_w$"" [-]")
ylabel(L"$p_c$"" [Pa]")
legend(["Drainage", "Imbibition"])
savefig("Capillary Pressure/method1.png")

figure()
sw_plot = collect(linspace(0,1,1000))
plot(sw_plot, pc_drain2.(sw_plot, pce, swc), sw_plot, pc_imb2.(sw_plot, pce, swc, sor))
title("Capillary Pressure, "L"$p_c$"" [Pa]")
xlabel(L"$S_w$"" [-]")
ylabel(L"$p_c$"" [Pa]")
legend(["Drainage", "Imbibition"])
savefig("Capillary Pressure/method2.png")

figure()
sw_plot = collect(linspace(0,1,1000))
plot(sw_plot, pc_drain3.(sw_plot, pce, swc), sw_plot, pc_imb3.(sw_plot, pce, swc, sor))
title("Capillary Pressure, "L"$p_c$"" [Pa]")
xlabel(L"$S_w$"" [-]")
ylabel(L"$p_c$"" [Pa]")
legend(["Drainage", "Imbibition"])
savefig("Capillary Pressure/method3.png")

#Van-Genuchten
a_drain = 0.066
a_imb = 0.6
m_drain = 0.6
m_imb = 0.5
M_drain = 0.26
M_imb = 0.2

k = 1e-12
ϕ = 0.4
sigma = 25.0*1000 #N/m
theta = 0.785

J = (S, S_wc, S_or, a, m, M) -> a.*(((S-S_wc)./(1.0-S_or-S_wc)).^(-1.0./m)-1.0).^(1.0-m)-M
p_c = (S, S_wc, S_or, a, m, M) -> J.(S, S_wc, S_or, a, m, M).*sigma.*cos(theta).*sqrt(ϕ/k)

sw = collect(linspace(swc+0.02,1.0-sor,1000))

figure()
plot(sw, p_c.(sw, swc, sor, a_drain, m_drain, M_drain), sw, p_c.(sw, swc, sor, a_imb, m_imb, M_imb))
title("Capillary Pressure, "L"$p_c$"" [Pa]")
xlabel(L"$S_w$"" [-]")
ylabel(L"$p_c$"" [Pa]")
legend(["Drainage", "Imbibition"])
savefig("Capillary Pressure/Van_Genuchten.png")


# f = (x,n) -> (x^2 - 1) % n
# println(f(3, 5))

# function gcd(a,b)
# 	while b != 0
# 		a, b = b, (a % b)
# 	end
# 	a
# end

# function pollardrho(n)
# 	f = (x,n) -> (x^2 - 1) % n
# 	x, y, d = 2, 2, 1
# 	while d == 1
# 		x = f(x,n)
# 		y = f(f(y,n),n)
# 		d = gcd(abs(x-y), n)
# 	end
# 	d
# end

function isprime(n)
	if n & 1 == 0
		return false
	end
	for i = 3:sqrt(n)
		if n % i == 0
			return false
		end
	end
	return true
end

N = 600851475143
largest_factor = 0

for candidate = 3:convert(Int64, ceil(sqrt(N)))
	if N % candidate == 0
		if isprime(candidate)
			largest_factor = candidate
		elseif isprime(convert(Int64, N / candidate))
			largest_factor = convert(Int64, N / candidate)
		end
	end
end
println("The answer to Euler Problem 3 is $largest_factor")


"""
    wernerState(dim :: Int, p ::Union{Int,Float64}) :: Matrix{Float64}

This function constructs the Werner states,
```math
    \\sigma_{d,p} = p \\frac{\\Pi_{0}}{d+1 \\choose 2} + (1-p) \\frac{\\Pi_{1}}{d \\choose 2}
```
where ``p \\in [0,1]`` and ``\\Pi_0, \\Pi_1`` are the projectors onto the symmetric and anti-symmetric
subspaces respectively. They can be determined by
```math
    \\Pi_0 = \\frac{1}{2} (I_{A} \\otimes I_{B} + \\mathbb{F}) \\hspace{1cm} \\Pi_1 = \\frac{1}{2}(I_{A} \\otimes I_{B} - \\mathbb{F})
```
where ``\\mathbb{F}`` is the swap operator.

A `DomainError` is thrown if:
* `d ≤ 1`
* `p` is not in range `0 ≤ p ≤ 1`
"""
function wernerState(d :: Int, p ::Union{Int,Float64}) :: Matrix{Float64}
    if d ≤ 1
        throw(DomainError(d,"wernerState requires the local dimension is two or greater."))
    elseif !(0 ≤ p ≤ 1)
        throw(DomainError(p,"wernerState requires p ∈ [0,1]."))
    end
    swap = swapOperator(d)
    Π0 = (I + swap)/2
    Π1 = (I - swap)/2
    return p * Π0 / binomial(d+1,2) + (1-p) * Π1 / binomial(d,2)
end

"""
    axisymmetricState(
        d :: Int64,
        x :: Union{Int,Float64},
        y :: Union{Int,Float64}
    ) :: Matrix{Float64}

Construct the axisymmetric state ``\\rho^{\\text{axi}}`` as described in section
IV.C. of [this paper](https://arxiv.org/pdf/1505.01833.pdf).
This state is a bipartite quantum state with each subspace having dimension `d`.
The diagonal of ``\\rho^{\\text{axi}}`` is parameterized as

```math
    \\rho^{\\text{axi}}_{jj,jj} = \\frac{1}{d^2}+a, \\quad \\text{and}\\quad \\rho^{\\text{axi}}_{jk,jk} = \\frac{1}{d^2}- \\frac{a}{d-1} \\; (j\\neq k)
```

while the off-diagonals are expressed as

```math
    \\rho^{\\text{axi}}_{jj,kk} = b,
```

where ``a = y\\frac{d-1}{d}`` and ``b = \\frac{x}{\\sqrt{d(d-1)}}``.
All remaining elements of ``\\rho^{\\text{axi}}`` are zero.
Inputs `x` and `y` of the `axisymmetricState` function parameterize ``\\rho^{\\text{axi}}``
and are constrained as:
* ``-\\frac{1}{d\\sqrt{d-1}} \\leq y \\leq \\frac{\\sqrt{d-1}}{d}``
* ``-\\frac{1}{\\sqrt{d(d-1)}}\\leq x \\leq \\sqrt{\\frac{d-1}{d}}``
* ``-\\frac{1}{\\sqrt{d}}\\left( y + \\frac{1}{d\\sqrt{d-1}}\\right) \\leq x \\leq \\frac{d-1}{\\sqrt{d}}\\left(y + \\frac{1}{d\\sqrt{d-1}} \\right)``

If any of the constraints above do not hold, a `DomainError` is thrown.
"""
function axisymmetricState(
    d :: Int64,
    x :: Union{Int,Float64},
    y :: Union{Int,Float64}
) :: Matrix{Float64}
    y_bounds = _axisymmetric_y_bounds(d)
    x_bounds = _axisymmetric_x_bounds(d)
    x_constraints = _axisymmetric_x_constraints(d,y)

    if !(y_bounds[1] ≤ y ≤ y_bounds[2])
        throw(DomainError(y, "input `y` does not satisfy `$(y_bounds[1]) ≤ y ≤ $(y_bounds[2])`."))
    elseif !(x_bounds[1] ≤ x ≤ x_bounds[2])
        throw(DomainError(x, "input `x` does not satisfy `$(x_bounds[1]) ≤ x ≤ $(x_bounds[2])`."))
    elseif !(x_constraints[1] ≤ x ≤ x_constraints[2])
        throw(DomainError(x, "input `x` does not satisfy `$(x_constraints[1]) ≤ x ≤ $(x_constraints[2])`."))
    elseif d ≤ 1
        throw(DomainError(d, "input `d` must satisfy `d ≥ 2`."))
    end

    a = y*sqrt(d-1)/d
    b = x/sqrt(d*(d-1))

    ρ_axi = zeros(Float64, d, d, d, d)
    for j in 1:d, k in 1:d
        if j == k
            ρ_axi[j,j,j,j] = a + 1/d^2
        else
            ρ_axi[j,k,j,k] = 1/d^2 - a/(d-1)
            ρ_axi[j,j,k,k] = b
        end
    end

    return reshape(ρ_axi, d^2, d^2)
end

# Helper for axisymmetricState, implements Eq. (42a) of https://arxiv.org/pdf/1505.01833.pdf
function _axisymmetric_y_bounds(d :: Int64) :: Tuple{Float64, Float64}
    return (-1/(d*sqrt(d-1)), sqrt(d-1)/d)
end

# Helper for axisymmetricState, implements Eq. (42b) of https://arxiv.org/pdf/1505.01833.pdf
function _axisymmetric_x_bounds(d :: Int64) :: Tuple{Float64, Float64}
    return (-1/sqrt(d*(d-1)), sqrt((d-1)/d))
end

# Helper for axisymmetricState, implements Eq. (43) of https://arxiv.org/pdf/1505.01833.pdf
function _axisymmetric_x_constraints(d,y) :: Tuple{Float64, Float64}
    lower = -1/sqrt(d)*(y + 1/(d*sqrt(d-1)))
    upper = (d-1)/sqrt(d)*(y+1/(d*sqrt(d-1)))

    return (lower, upper)
end

"""
    haarStates(n :: Int64, d :: Int64) :: Vector{Matrix{ComplexF64}}

Constructs a list of `n` random states on a `d`-dimensional Hilbert space
according to the Haar measure.
"""
function haarStates(n :: Int64, d :: Int64) :: Vector{Matrix{ComplexF64}}
    ρ0 = zeros(Float64, d, d)
    ρ0[1,1] = 1.0

    rand_unitaries = map(i -> random_unitary(d), 1:n)

    return map(U -> U*ρ0*U', rand_unitaries)
end


# This function calculates the free kurtosis of a matrix
# The function is uniform for both Hermitian or rectangular X
"""
    κ₄(X)

This function return the free kurtosis of a matrix X

# Arguments
-   `X`: a matrix, Hermitian or rectangular

# Outputs
-   `fk`: a scalar, the free kurtosis of X
"""
function κ₄(X)
    # get dimension
    N, M = size(X)

    # compute the free kurtosis
    fk = tr((X*X')^2)/N - (1 + N/M) * (tr(X*X')/N)^2
    return fk
end

"""
    neg_abs_sum_free_kurt(Z)

This function calculates the sum of negative absolute value of free kurtosis of components of Z
"""
function neg_abs_sum_free_kurt(Z)
    #--------------------------------------------------------------------------
    # Syntax:       F = neg_abs_sum_free_kurt(Z)
    #
    # Input:        Z: a array of Hermitian or rectangular matrices
    #
    # Outputs:      F: a scalr, it is the summation of negative absolute values
    #               of the free kurtosis of Z[i]
    #
    # Description:  This function return the summation of negative absolute value
    #               of free kurtosis of input array of matrices
    #
    # Authors:      Hao Wu
    #               lingluanwh@gmail.com
    #
    # Date:         Jan 20, 2019
    #--------------------------------------------------------------------------

    return -sum(abs.(κ₄.(Z)))
end

"""
    free_ent(X; mat = "her")

This function return the free entropy of a matrix X

# Arguments
-   `X`: a matrix, Hermitian or rectangular
-   `mat`: the type of X, valid options are "her" and "rec"

# Outputs
-   `chi`: a scalr, it is the free entropy of X
"""
function free_ent(X; mat = "her")
    # get dimension
    N, M = size(X)

    # compute the entropy
    # self-adjoint case
    if mat == "her"
        Λ = eigvals(Hermitian(X))
        chi = mean(log.(abs.([Λ[i] - Λ[j] for i = 1:N for j = (i + 1):N])))
    end

    # rectangular case
    if mat == "rec"
        a, b = N/(N + M), M/(N + M)
        Λ = svdvals(X*X')
        chi = a^2 * mean(log.(abs.([Λ[i] - Λ[j] for i = 1:N for j = (i + 1):N]))) + a*(b - a)*mean(log.(abs.(Λ)))
    end

    return chi
end

"""
    sum_free_ent(Z; mat = "her")

This function calculates the sum of free entropy of components of Z
"""
function sum_free_ent(Z; mat = "her")
    return sum(free_ent.(Z, mat = mat))
end


using DifferentialEquations
f(u,p,t) = 0.98u
u0 = 1.0
tspan = (0.0,1.0)
prob = ODEProblem(f,u0,tspan)

sol = solve(prob)

using Plots; gr()
plot(sol)

plot(sol,linewidth=5,title="Solution to the linear ODE with a thick line",
     xaxis="Time (t)",yaxis="u(t) (in μm)",label="My Thick Line!") ## legend=false

plot!(sol.t, t->1.0*exp(0.98t),lw=3,ls=:dash,label="True Solution!")

sol.t

sol.u

[t+u for (u,t) in tuples(sol)]

sol

sol(0.45)

sol = solve(prob,abstol=1e-8,reltol=1e-8)

plot(sol)
plot!(sol.t, t->1.0*exp(0.98t),lw=3,ls=:dash,label="True Solution!")

sol = solve(prob,saveat=0.1)

sol = solve(prob,saveat=[0.2,0.7,0.9])

sol = solve(prob,saveat=[0.2,0.7,0.9],save_start = false, save_end = false)

sol = solve(prob,dense=false)

sol = solve(prob,save_everystep=false)

sol = solve(prob,alg_hints=[:stiff])

sol = solve(prob,Tsit5(),reltol=1e-6)

function lorenz!(du,u,p,t)
    σ,ρ,β = p
    du[1] = σ*(u[2]-u[1])
    du[2] = u[1]*(ρ-u[3]) - u[2]
    du[3] = u[1]*u[2] - β*u[3]
end

u0 = [1.0,0.0,0.0]

p = (10,28,8/3) ## we could also make this an array, or any other type!

tspan = (0.0,100.0)
prob = ODEProblem(lorenz!,u0,tspan,p)

sol = solve(prob)

sol.t[10],sol[10]

sol[2,10]

A = convert(Array,sol)

plot(sol)

plot(sol,vars=(1,2,3))

plot(sol,vars=(1,2,3),denseplot=false)

plot(sol,vars=(0,2))

function lotka_volterra!(du,u,p,t)
  du[1] = p[1]*u[1] - p[2]*u[1]*u[2]
  du[2] = -p[3]*u[2] + p[4]*u[1]*u[2]
end

lv! = @ode_def LotkaVolterra begin
  dx = a*x - b*x*y
  dy = -c*y + d*x*y
end a b c d

u0 = [1.0,1.0]
p = (1.5,1.0,3.0,1.0)
tspan = (0.0,10.0)
prob = ODEProblem(lv!,u0,tspan,p)
sol = solve(prob)
plot(sol)

lv!.Jex

A  = [1. 0  0 -5
      4 -2  4 -3
     -4  0  0  1
      5 -2  2  3]
u0 = rand(4,2)
tspan = (0.0,1.0)
f(u,p,t) = A*u
prob = ODEProblem(f,u0,tspan)
sol = solve(prob)

sol[3]

big_u0 = big.(u0)

prob = ODEProblem(f,big_u0,tspan)
sol = solve(prob)

sol[1,3]

prob = ODEProblem(f,big_u0,big.(tspan))
sol = solve(prob)

using StaticArrays
A  = @SMatrix [ 1.0  0.0 0.0 -5.0
                4.0 -2.0 4.0 -3.0
               -4.0  0.0 0.0  1.0
                5.0 -2.0 2.0  3.0]
u0 = @SMatrix rand(4,2)
tspan = (0.0,1.0)
f(u,p,t) = A*u
prob = ODEProblem(f,u0,tspan)
sol = solve(prob)

sol[3]

# This file was generated using Literate.jl, https://github.com/fredrikekre/Literate.jl


# determinant

function Fdet(x)
    n = Int(sqrt(length(x)))
    if n==2
        return(x[1,1]*x[2,2] - x[1,2]*x[2,1])
    elseif isodd(n)
        Sign = repeat([1,-1],Int(ceil(n/2)))[1:n].*x[1,:]

        #println(s)
    else
        Sign = repeat([1,-1],Int(round(n/2))).*x[1,:]
    end
    s=0
    for i in 1:n
        s+=Sign[i] * Fdet(x[2:n,append!(collect(1:(i-1)),collect((i+1):n))])
    end
    return s
end

Fdet([10 20 10;4 5 6;2 3 5])

Fdet([1 4;2 3])


using StaticArrays
function cspline(s, t1, t2, p1, p2, m1, m2)
    d = t2-t1
    t = (s-t1)/(t2-t1)
    t2 = t*t
    t3 = t2*t
    z1, z2, z3, z4 = (1 - 3*t2 + 2*t3,  3*t2 - 2*t3,  t - 2*t2 + t3, 0 - t2 + t3)
    z1*p1 + z2*p2 + z3*d*m1 + z4*d*m2
end
function intcspline(s, t1, t2, p1, p2, m1, m2)
        d = t2-t1
        t = (s-t1)/(t2-t1)
        t2 = t*t
        t3 = t2*t
        t4 = t2*t2
        t4, t3, t2 = t4/4, t3/3, t2/2
        z1, z2, z3, z4 = (t - 3*t3 + 2*t4,  3*t3 - 2*t4,  t2 - 2*t3 + t4, 0 - t3 + t4)
        (z1*p1 + z2*p2 + z3*d*m1 + z4*d*m2)*d
end
intcspline(s, T, t1, t2, p1, p2, m1, m2) = intcspline(T, t1, t2, p1, p2, m1, m2) - intcspline(s, t1, t2, p1, p2, m1, m2)

struct CSpline{T}
    s; t; x::T; y::T; mx; my
end

"""
    CSpline(s, t, x, y = x, m0 = (y-x)/(t-s), m1 = (y-x)/(t-s))

Cubic spline parametrized by ``f(s) = x`` and ``f(t) = y``, ``f'(s) = m_0``, ``f'(t) = m_1``.
"""
CSpline(s, t, x::T, y = x, m0 = (y-x)/(t-s), m1 =  (y-x)/(t-s)) where {T} = CSpline{T}(s, t, x, y, mx, my)
(cs::CSpline)(t) =  cspline(t, cs.s, cs.t, cs.x, cs.y, cs.mx, cs.my)

"""
    integrate(cs::CSpline, s, t)

Integrate the cubic spline from `s` to `t`.
"""
integrate(cs::CSpline, s, t) = intcspline(s,t, cs.s, cs.t, cs.x, cs.y, cs.mx, cs.my)


versioninfo()

# If needed:
#Pkg.add("DifferentialEquations")
#Pkg.add("PyPlot")
#Pkg.add("Plots")

using Plots
gr()
using DifferentialEquations

b = 0.25
c = 5.0

y0 = [pi - 0.1; 0.0]

function pend(t, y, dy)
    dy[1] = y[2]
    dy[2] = (-b * y[2]) - (c * sin(y[1]))
end

function f_pend(y, t)
    return [y[2], (-b * y[2]) - (c * sin(y[1]))]
end

tspan = (0.0, 10.0)

function odeint_1(f, y0, tspan)
    prob = ODEProblem(f, y0, tspan)
    sol = solve(prob)
    return sol.t, hcat(sol.u...)'
end

function odeint(f, y0, tspan)
    t, sol = odeint_1(f, y0, tspan)
    return sol
end

t, sol = odeint_1(pend, y0, tspan)

plot(t, sol[:, 1], xaxis="Time t", title="Solution to the pendulum ODE", label="\\theta (t)")
plot!(t, sol[:, 2], label="\\omega (t)")

function rungekutta1(f, y0, t)
    n = length(t)
    y = zeros((n, length(y0)))
    y[1,:] = y0
    for i in 1:n-1
        h = t[i+1] - t[i]
        y[i+1,:] = y[i,:] + h * f(y[i,:], t[i])
    end
    return y
end

t = linspace(0, 10, 101);
sol = rungekutta1(f_pend, y0, t);

plot(t, sol[:, 1], xaxis="Time t", title="Solution to the pendulum ODE with Runge-Kutta 1", label="\\theta (t)")
plot!(t, sol[:, 2], label="\\omega (t)")

t2 = linspace(0, 10, 1001);
sol2 = rungekutta1(f_pend, y0, t2);

plot(t2, sol2[:, 1], xaxis="Time t", title="Solution to the pendulum ODE with Runge-Kutta 1", label="\\theta (t)")
plot!(t2, sol2[:, 2], label="\\omega (t)")

t3 = linspace(0, 10, 2001);
sol3 = rungekutta1(f_pend, y0, t3);

plot(t3, sol3[:, 1], xaxis="Time t", title="Solution to the pendulum ODE with Runge-Kutta 1", label="\\theta (t)")
plot!(t3, sol3[:, 2], label="\\omega (t)")

function rungekutta2(f, y0, t)
    n = length(t)
    y = zeros((n, length(y0)))
    y[1,:] = y0
    for i in 1:n-1
        h = t[i+1] - t[i]
        y[i+1,:] = y[i,:] + h * f(y[i,:] + f(y[i,:], t[i]) * h/2, t[i] + h/2)
    end
    return y
end

t3 = linspace(0, 10, 21);
sol3 = rungekutta2(f_pend, y0, t3);

plot(t3, sol3[:, 1], xaxis="Time t", title="Solution to the pendulum ODE with Runge-Kutta 2 (21 points)", label="\\theta (t)")
plot!(t3, sol3[:, 2], label="\\omega (t)")

t3 = linspace(0, 10, 101);
sol3 = rungekutta2(f_pend, y0, t3);

plot(t3, sol3[:, 1], xaxis="Time t", title="Solution to the pendulum ODE with Runge-Kutta 2 (101 points)", label="\\theta (t)")
plot!(t3, sol3[:, 2], label="\\omega (t)")

function rungekutta4(f, y0, t)
    n = length(t)
    y = zeros((n, length(y0)))
    y[1,:] = y0
    for i in 1:n-1
        h = t[i+1] - t[i]
        k1 = f(y[i,:], t[i])
        k2 = f(y[i,:] + k1 * h/2, t[i] + h/2)
        k3 = f(y[i,:] + k2 * h/2, t[i] + h/2)
        k4 = f(y[i,:] + k3 * h, t[i] + h)
        y[i+1,:] = y[i,:] + (h/6) * (k1 + 2*k2 + 2*k3 + k4)
    end
    return y
end

t = linspace(0, 10, 31);
sol = rungekutta4(f_pend, y0, t);

plot(t, sol[:, 1], xaxis="Time t", title="Solution to the pendulum ODE with Runge-Kutta 4 (31 points)", label="\\theta (t)")
plot!(t, sol[:, 2], label="\\omega (t)")

t = linspace(0, 10, 101);
sol = rungekutta4(f_pend, y0, t);

plot(t, sol[:, 1], xaxis="Time t", title="Solution to the pendulum ODE with Runge-Kutta 4 (101 points)", label="\\theta (t)")
plot!(t, sol[:, 2], label="\\omega (t)")

methods = [rungekutta1, rungekutta2, rungekutta4]
markers = [:o, :s, :>]

function test_1(n=101)
    t = linspace(0, 10, n)
    tspan = (0.0, 10.0)
    t1, sol = odeint_1(pend, y0, tspan)
    plt = plot(t1, sol[:, 1], marker=:d, xaxis="Time t", title="Solution to the pendulum ODE with ($n points)", label="odeint")
    for (method, m) in zip(methods, markers)
        sol = method(f_pend, y0, t)
        plot!(t, sol[:, 1], marker=m, label=string(method))
    end
    display(plt)
end

test_1(10)

test_1(20)

test_1(100)

test_1(200)

y0 = [0; 1; 0.1]

function f(y, t)
    return [y[2], y[3], 12 * y[1]^(4/5) + cos(y[2])^3 - sin(y[3])]
end

function f_2(t, y, dy)
    dy[1] = y[2]
    dy[2] = y[3]
    dy[3] = 12 * y[1]^(4/5) + cos(y[2])^3 - sin(y[3])
end

function test_2(n=101)
    t = linspace(0, 1, n)
    tspan = (0.0, 1.0)
    t1, sol = odeint_1(f_2, y0, tspan)
    plt = plot(t1, sol[:, 1], marker=:d, xaxis="Time t", title="Solution to an ODE with ($n points)", label="odeint")
    for (method, m) in zip(methods, markers)
        sol = method(f, y0, t)
        plot!(t, sol[:, 1], marker=m, label=string(method))
    end
    display(plt)
end

test_2(10)

test_2(50)

y0 = [10.0, -3.0, 1.0, 1.0]

function f(y, t)
    return [y[2], y[3], y[4], y[1]^(-5/3)]
end

function f_2(t, y, dy)
    dy[1] = y[2]
    dy[2] = y[3]
    dy[3] = y[4]
    dy[4] = y[1]^(-5/3)
end

function test_3(n=101)
    t = linspace(0, 1, n)
    tspan = (0.0, 1.0)
    t1, sol = odeint_1(f_2, y0, tspan)
    plt = plot(t1, sol[:, 1], marker=:d, xaxis="Time t", title="Solution to an ODE with ($n points)", label="odeint")
    for (method, m) in zip(methods, markers)
        sol = method(f, y0, t)
        plot!(t, sol[:, 1], marker=m, label=string(method))
    end
    display(plt)
end

test_3(10)

test_3(50)

function benchmark(n=101)
    t = linspace(0, 1, n)
    tspan = (0.0, 1.0)

    print("Time of solving an ODE with the 'solve' method from 'DifferentialEquations' ...\n")
    @time t1, sol = odeint_1(f_2, y0, tspan)
    for method in methods
        print("Time of solving an ODE with the $method method for $n points ...\n")
        @time sol = method(f, y0, t)
    end
end

for n in [20, 100, 1000]
    print("\nFor $n points...\n")
    benchmark(n)
end

using BenchmarkTools, Compat

function benchmark(n=101)
    t = linspace(0, 1, n)
    tspan = (0.0, 1.0)

    print("Time of solving an ODE with the 'solve' method from 'DifferentialEquations' ...\n")
    @btime t1, sol = $odeint_1($f_2, $y0, $tspan)
    for method in methods
        print("Time of solving an ODE with the $method method for $n points ...\n")
        @btime sol = $method($f, $y0, $t)
    end
end

for n in [20, 100, 1000]
    print("\nFor $n points...\n")
    benchmark(n)
end


"""
    legendre(x, p::Integer)

Evaluate the Legendre polynomial of degree `p` at `x` using the three term
recursion [Kopriva, Implementing Spectral Methods for PDEs, Algorithm 20].
"""
function legendre(x, p::Integer)
    a::typeof(x) = one(x)
    b::typeof(x) = x

    if p <= 0
        return a
    elseif p == 1
        return b
    end

    for j in 2:p
        a, b = b, ( (2j-1)*x*b - (j-1)*a ) / j
    end

    b
end

"""
    legendre_and_derivative(x, p::Integer)

Evaluate the Legendre polynomial of degree `p` and its derivative at `x` using
the three term recursion [Kopriva, Implementing Spectral Methods for PDEs,
Algorithm 22].
"""
function legendre_and_derivative(x, p::Integer)
    # coefficients for the polynomial...
    a::typeof(x) = one(x)
    b::typeof(x) = x
    # ... and for the derivative
    aa::typeof(x) = zero(x)
    bb::typeof(x) = one(x)

    if p <= 0
        return a, aa
    elseif p == 1
        return b, bb
    end

    for j in 2:p
        a, b = b, ( (2j-1)*x*b - (j-1)*a ) / j
        aa, bb = bb, aa + (2j-1)*a
    end

    b, bb
end

"""
    gauss_legendre_nodes_and_weights(p, T=Float64::Type, tol=4*eps(T), maxit=100)

Compute the Gauss-Legendre nodes and weights for polynomials of degree `p`
using the scalar type `T`, tolerance `tol` and maximal number of Newton iterations
`maxit` [Kopriva, Implementing Spectral Methods for PDEs, Algorithm 23].
"""
function gauss_legendre_nodes_and_weights(p, T=Float64::Type, tol=4*eps(T), maxit=100)
    if p <= 0
        return T[0], T[2]
    elseif p == 1
        x0 = -1 / sqrt(T(3))
        return [x0, -x0], T[1, 1]
    end

    nodes = Vector{T}(undef, p+1)
    weights = Vector{T}(undef, p+1)
    for j in 0:((p+1)÷2-1)
        x = -cospi(T(2j+1)/(2p+2))
        for k in 1:maxit
            pol, der = legendre_and_derivative(x, p+1)
            Δ = -pol / der
            x = x + Δ
            abs(Δ) <= tol*abs(x) && break
        end
        pol, der = legendre_and_derivative(x, p+1)
        nodes[j+1] = x
        nodes[p-j+1] = -x
        weights[j+1] = weights[p-j+1] = 2 / ( (1-x^2)*der^2 )
    end

    if mod(p,2) == 0
        pol, der = legendre_and_derivative(zero(T), p+1)
        nodes[p÷2+1]   = 0
        weights[p÷2+1] = 2 / der^2
    end

    return nodes, weights
end

# helper function [Kopriva, Implementing Spectral Methods for PDEs, Algorithm 24]
function q_and_L_evaluation(x, p::Integer)
    a = one(x)
    b = x
    aa = zero(x)
    bb = one(x)

    for j in 2:p
        a, b = b, ( (2j-1)*x*b - (j-1)*a ) / j
        aa, bb = bb, aa + (2j-1)*a
    end
    pol = ( (2p+1)*x*b - p*a ) / (p+1)
    der = aa + (2p+1)*b

    pol-a, der-aa, b
end

"""
    lobatto_legendre_nodes_and_weights(p, T=Float64::Type, tol=4*eps(T), maxit=100)

Compute the Lobatto-Legendre nodes and weights for polynomials of degree `p`
using the scalar type `T`, tolerance `tol` and maximal number of Newton iterations
`maxit` [Kopriva, Implementing Spectral Methods for PDEs, Algorithm 25].
"""
function lobatto_legendre_nodes_and_weights(p, T=Float64::Type, tol=4*eps(T), maxit=100)
    if p <= 0
        return T[0], T[2]
    elseif p == 1
        return T[-1, 1], T[1, 1]
    end

    nodes = Vector{T}(undef, p+1)
    weights = Vector{T}(undef, p+1)

    nodes[1] = -1
    nodes[end] = 1
    weights[1] = weights[end] = T(2) / (p*(p+1))

    for j in 1:((p+1)÷2-1)
        x = -cos( T(j+1//4)*π/p - 3/(8p*T(j+1//4)*π) )
        for k in 1:maxit
            pol, der, leg = q_and_L_evaluation(x, p)
            Δ = -pol / der
            x = x + Δ
            abs(Δ) <= tol*abs(x) && break
        end
        pol, der, leg = q_and_L_evaluation(x, p)
        nodes[j+1] = x
        nodes[p-j+1] = -x
        weights[j+1] = weights[p-j+1] = 2 / ( p*(p+1)*leg^2 )
    end

    if mod(p,2) == 0
        pol, der, leg = q_and_L_evaluation(zero(T), p)
        nodes[p÷2+1]   = 0
        weights[p÷2+1] = 2 / ( p*(p+1)*leg^2 )
    end

    return nodes, weights
end

"""
    legendre_vandermonde(nodes)

Computes the Vandermonde matrix with respect to the Legendre polynomials and
the nodal basis on `nodes`.
The Vandermonde matrix `V` is the transformation matrix from the modal Legendre
basis to the nodal Lagrange basis associated with `nodes`.
"""
function legendre_vandermonde(nodes::AbstractVector)
    T = eltype(nodes)
    pp1 = length(nodes)
    V = Array{T}(undef, pp1, pp1)
    for j in 1:pp1, (i,x) in enumerate(nodes)
      V[i, j] = legendre(x, j-1)
    end
    V
end

legendre_vandermonde(basis::NodalBasis{Line}) = legendre_vandermonde(basis.nodes)

"""
    legendre_M(p, T=Float64)

Computes the diagonal mass matrix in the modal Legendre basis up to degree `p`
using the scalar type `T`.
"""
function legendre_M(p, T=Float64)
  Diagonal( T[2//(2n+1) for n in 0:p] )
end

"""
    legendre_D(p, T=Float64)

Computes the derivative matrix in the modal Legendre basis up to degree `p`
using the scalar type `T`.
"""
function legendre_D(p, T=Float64)
  D = fill(zero(T), p+1, p+1)
  if p >= 1
    D[1, 2] = 1
  end
  if p >= 2
    D[2, 3] = 3
  end
  for col in 4:p+1
    D[:, col] = D[:, col-2]
    D[col-1, col] = 2*col-3
  end
  D
end