Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sparsehessian fails when getindex used #555

Closed
odow opened this issue Mar 21, 2022 · 1 comment · Fixed by #1384
Closed

sparsehessian fails when getindex used #555

odow opened this issue Mar 21, 2022 · 1 comment · Fixed by #1384

Comments

@odow
Copy link
Contributor

odow commented Mar 21, 2022 •
edited by sync-by-unito bot
Loading

This is on the latest master of Symbolics.jl:

(Symbolics) pkg> st
     Project Symbolics v4.3.1
      Status `~/.julia/dev/Symbolics/Project.toml`
  [4fba245c] ArrayInterface v5.0.5
  [187b0558] ConstructionBase v1.3.0
  [864edb3b] DataStructures v0.18.11
  [b552c78f] DiffRules v1.10.0
  [31c24e10] Distributions v0.25.52
  [ffbed154] DocStringExtensions v0.8.6
  [5b8099bc] DomainSets v0.5.9
  [0b43b601] Groebner v0.1.1
  [615f187c] IfElse v0.1.1
  [23fbe1c1] Latexify v0.15.13
  [1914dd2f] MacroTools v0.5.9
  [e9d8d322] Metatheory v1.3.3
  [77ba4419] NaNMath v0.3.7
  [3cdcf5f2] RecipesBase v1.2.1
  [189a3867] Reexport v1.2.2
  [ae029012] Requires v1.3.0
  [7e49a35a] RuntimeGeneratedFunctions v0.5.3
  [0bca4576] SciMLBase v1.28.0
  [efcf1570] Setfield v0.8.2
  [276daf66] SpecialFunctions v2.1.4
  [90137ffa] StaticArrays v1.4.2
  [d1185830] SymbolicUtils v0.19.7
  [8ea1fca8] TermInterface v0.2.3
  [a2a6695c] TreeViews v0.3.0
  [8f399da3] Libdl
  [37e2e46d] LinearAlgebra
  [2f01184e] SparseArrays

julia> import Symbolics
[ Info: Precompiling Symbolics [0c5d862f-8b57-4792-8d23-62f2024744c7]

julia> Symbolics.@variables(p[1:1], x[1:1])
2-element Vector{Symbolics.Arr{Symbolics.Num, 1}}:
 p[1:1]
 x[1:1]

julia> p, x = collect(p), collect(x)
(Symbolics.Num[p[1]], Symbolics.Num[x[1]])

julia> Symbolics.sparsehessian(p[1] * x[1], x)
ERROR: Failed to apply rule (~f)(~x, ~y) => begin
        #= /Users/oscar/.julia/dev/Symbolics/src/diff.jl:556 =#
        if haslinearity_2(~f)
            #= /Users/oscar/.julia/dev/Symbolics/src/diff.jl:557 =#
            a = if isidx(~x)
                    ~x
                else
                    _scalar
                end
            #= /Users/oscar/.julia/dev/Symbolics/src/diff.jl:558 =#
            b = if isidx(~y)
                    ~y
                else
                    _scalar
                end
            #= /Users/oscar/.julia/dev/Symbolics/src/diff.jl:559 =#
            combine_terms_2(linearity_2(~f), a, b)
        else
            #= /Users/oscar/.julia/dev/Symbolics/src/diff.jl:561 =#
            error("Function of unknown linearity used: ", ~f)
        end
    end on expression 0[1]
Stacktrace:
  [1] (::Metatheory.Rules.DynamicRule)(term::SymbolicUtils.Term{Any, Nothing})
    @ Metatheory.Rules ~/.julia/packages/Metatheory/XcKKW/src/Rules.jl:234
  [2] (::Metatheory.Rewriters.Chain)(x::SymbolicUtils.Term{Any, Nothing})
    @ Metatheory.Rewriters ~/.julia/packages/Metatheory/XcKKW/src/Rewriters.jl:69
  [3] (::Metatheory.Rewriters.Walk{:post, Metatheory.Rewriters.Chain, typeof(Symbolics.basic_simterm), false})(x::SymbolicUtils.Term{Any, Nothing})
    @ Metatheory.Rewriters ~/.julia/packages/Metatheory/XcKKW/src/Rewriters.jl:200
  [4] (::Metatheory.Rewriters.PassThrough{Metatheory.Rewriters.Walk{:post, Metatheory.Rewriters.Chain, typeof(Symbolics.basic_simterm), false}})(x::SymbolicUtils.Term{Any, Nothing})
    @ Metatheory.Rewriters ~/.julia/packages/Metatheory/XcKKW/src/Rewriters.jl:188
  [5] iterate
    @ ./generator.jl:47 [inlined]
  [6] collect_to!(dest::Vector{Symbolics.TermCombination}, itr::Base.Generator{Vector{Any}, Metatheory.Rewriters.PassThrough{Metatheory.Rewriters.Walk{:post, Metatheory.Rewriters.Chain, typeof(Symbolics.basic_simterm), false}}}, offs::Int64, st::Int64)
    @ Base ./array.jl:724
  [7] collect_to_with_first!(dest::Vector{Symbolics.TermCombination}, v1::Symbolics.TermCombination, itr::Base.Generator{Vector{Any}, Metatheory.Rewriters.PassThrough{Metatheory.Rewriters.Walk{:post, Metatheory.Rewriters.Chain, typeof(Symbolics.basic_simterm), false}}}, st::Int64)
    @ Base ./array.jl:702
  [8] _collect(c::Vector{Any}, itr::Base.Generator{Vector{Any}, Metatheory.Rewriters.PassThrough{Metatheory.Rewriters.Walk{:post, Metatheory.Rewriters.Chain, typeof(Symbolics.basic_simterm), false}}}, #unused#::Base.EltypeUnknown, isz::Base.HasShape{1})
    @ Base ./array.jl:696
  [9] collect_similar
    @ ./array.jl:606 [inlined]
 [10] map(f::Metatheory.Rewriters.PassThrough{Metatheory.Rewriters.Walk{:post, Metatheory.Rewriters.Chain, typeof(Symbolics.basic_simterm), false}}, A::Vector{Any})
    @ Base ./abstractarray.jl:2294
 [11] (::Metatheory.Rewriters.Walk{:post, Metatheory.Rewriters.Chain, typeof(Symbolics.basic_simterm), false})(x::SymbolicUtils.Term{Any, Nothing})
    @ Metatheory.Rewriters ~/.julia/packages/Metatheory/XcKKW/src/Rewriters.jl:198
 [12] (::Metatheory.Rewriters.Fixpoint{Metatheory.Rewriters.Walk{:post, Metatheory.Rewriters.Chain, typeof(Symbolics.basic_simterm), false}})(x::SymbolicUtils.Term{Any, Nothing})
    @ Metatheory.Rewriters ~/.julia/packages/Metatheory/XcKKW/src/Rewriters.jl:118
 [13] hessian_sparsity(f::SymbolicUtils.Mul{Real, Int64, Dict{Any, Number}, Nothing}, u::Vector{SymbolicUtils.Term{Real, Nothing}})
    @ Symbolics ~/.julia/dev/Symbolics/src/diff.jl:576
 [14] sparsehessian(O::Symbolics.Num, vars::Vector{Symbolics.Num}; simplify::Bool)
    @ Symbolics ~/.julia/dev/Symbolics/src/diff.jl:608
 [15] sparsehessian(O::Symbolics.Num, vars::Vector{Symbolics.Num})
    @ Symbolics ~/.julia/dev/Symbolics/src/diff.jl:606
 [16] top-level scope
    @ REPL[10]:1

caused by: Function of unknown linearity used: getindex
Stacktrace:
  [1] error(::String, ::Function)
    @ Base ./error.jl:42
  [2] (::Symbolics.var"#166#175"{Symbolics.TermCombination})(_lhs_expr::SymbolicUtils.Term{Any, Nothing}, _subst::Base.ImmutableDict{Int64, Any}, _egraph::Nothing, f::Function, x::Int64, y::Int64)
    @ Symbolics ~/.julia/packages/Metatheory/XcKKW/src/Syntax.jl:343
  [3] success
    @ ~/.julia/packages/Metatheory/XcKKW/src/Rules.jl:228 [inlined]
  [4] (::Metatheory.var"#loop#28"{Metatheory.Rules.var"#success#5"{Metatheory.Rules.DynamicRule, SymbolicUtils.Term{Any, Nothing}}})(term::Metatheory.LL{Vector{Int64}}, bindings′::Base.ImmutableDict{Int64, Any}, matchers′::Metatheory.LL{Tuple{Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}, Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}, Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}}})
    @ Metatheory ~/.julia/packages/Metatheory/XcKKW/src/matchers.jl:133
  [5] (::Metatheory.var"#26#29"{Metatheory.LL{Vector{Int64}}, Metatheory.LL{Tuple{Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}, Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}, Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}}}})(b::Base.ImmutableDict{Int64, Any}, n::Int64)
    @ Metatheory ~/.julia/packages/Metatheory/XcKKW/src/matchers.jl:143
  [6] (::Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}})(next::Metatheory.var"#26#29"{Metatheory.LL{Vector{Int64}}, Metatheory.LL{Tuple{Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}, Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}, Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}}}}, data::Metatheory.LL{Vector{Int64}}, bindings::Base.ImmutableDict{Int64, Any})
    @ Metatheory ~/.julia/packages/Metatheory/XcKKW/src/matchers.jl:33
  [7] (::Metatheory.var"#loop#28"{Metatheory.Rules.var"#success#5"{Metatheory.Rules.DynamicRule, SymbolicUtils.Term{Any, Nothing}}})(term::Metatheory.LL{Vector{Int64}}, bindings′::Base.ImmutableDict{Int64, Any}, matchers′::Metatheory.LL{Tuple{Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}, Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}, Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}}})
    @ Metatheory ~/.julia/packages/Metatheory/XcKKW/src/matchers.jl:137
  [8] (::Metatheory.var"#26#29"{Metatheory.LL{Vector{Int64}}, Metatheory.LL{Tuple{Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}, Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}, Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}}}})(b::Base.ImmutableDict{Int64, Any}, n::Int64)
    @ Metatheory ~/.julia/packages/Metatheory/XcKKW/src/matchers.jl:143
  [9] (::Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}})(next::Metatheory.var"#26#29"{Metatheory.LL{Vector{Int64}}, Metatheory.LL{Tuple{Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}, Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}, Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}}}}, data::Metatheory.LL{Vector{Int64}}, bindings::Base.ImmutableDict{Int64, Any})
    @ Metatheory ~/.julia/packages/Metatheory/XcKKW/src/matchers.jl:33
 [10] (::Metatheory.var"#loop#28"{Metatheory.Rules.var"#success#5"{Metatheory.Rules.DynamicRule, SymbolicUtils.Term{Any, Nothing}}})(term::Metatheory.LL{Vector{Int64}}, bindings′::Base.ImmutableDict{Int64, Any}, matchers′::Metatheory.LL{Tuple{Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}, Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}, Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}}})
    @ Metatheory ~/.julia/packages/Metatheory/XcKKW/src/matchers.jl:137
 [11] (::Metatheory.var"#26#29"{SymbolicUtils.Term{Any, Nothing}, Tuple{Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}, Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}, Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}}})(b::Base.ImmutableDict{Int64, Any}, n::Int64)
    @ Metatheory ~/.julia/packages/Metatheory/XcKKW/src/matchers.jl:143
 [12] (::Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}})(next::Metatheory.var"#26#29"{SymbolicUtils.Term{Any, Nothing}, Tuple{Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}, Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}, Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}}}, data::SymbolicUtils.Term{Any, Nothing}, bindings::Base.ImmutableDict{Int64, Any})
    @ Metatheory ~/.julia/packages/Metatheory/XcKKW/src/matchers.jl:33
 [13] (::Metatheory.var"#loop#28"{Metatheory.Rules.var"#success#5"{Metatheory.Rules.DynamicRule, SymbolicUtils.Term{Any, Nothing}}})(term::SymbolicUtils.Term{Any, Nothing}, bindings′::Base.ImmutableDict{Int64, Any}, matchers′::Tuple{Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}, Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}, Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}})
    @ Metatheory ~/.julia/packages/Metatheory/XcKKW/src/matchers.jl:137
 [14] (::Metatheory.var"#term_matcher#27"{Tuple{Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}, Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}, Metatheory.var"#slot_matcher#19"{Metatheory.Patterns.PatVar{typeof(Metatheory.alwaystrue)}}}})(success::Metatheory.Rules.var"#success#5"{Metatheory.Rules.DynamicRule, SymbolicUtils.Term{Any, Nothing}}, data::Tuple{SymbolicUtils.Term{Any, Nothing}}, bindings::Base.ImmutableDict{Int64, Any})
    @ Metatheory ~/.julia/packages/Metatheory/XcKKW/src/matchers.jl:147
 [15] (::Metatheory.Rules.DynamicRule)(term::SymbolicUtils.Term{Any, Nothing})
    @ Metatheory.Rules ~/.julia/packages/Metatheory/XcKKW/src/Rules.jl:232
 [16] (::Metatheory.Rewriters.Chain)(x::SymbolicUtils.Term{Any, Nothing})
    @ Metatheory.Rewriters ~/.julia/packages/Metatheory/XcKKW/src/Rewriters.jl:69
 [17] (::Metatheory.Rewriters.Walk{:post, Metatheory.Rewriters.Chain, typeof(Symbolics.basic_simterm), false})(x::SymbolicUtils.Term{Any, Nothing})
    @ Metatheory.Rewriters ~/.julia/packages/Metatheory/XcKKW/src/Rewriters.jl:200
 [18] (::Metatheory.Rewriters.PassThrough{Metatheory.Rewriters.Walk{:post, Metatheory.Rewriters.Chain, typeof(Symbolics.basic_simterm), false}})(x::SymbolicUtils.Term{Any, Nothing})
    @ Metatheory.Rewriters ~/.julia/packages/Metatheory/XcKKW/src/Rewriters.jl:188
 [19] iterate
    @ ./generator.jl:47 [inlined]
 [20] collect_to!(dest::Vector{Symbolics.TermCombination}, itr::Base.Generator{Vector{Any}, Metatheory.Rewriters.PassThrough{Metatheory.Rewriters.Walk{:post, Metatheory.Rewriters.Chain, typeof(Symbolics.basic_simterm), false}}}, offs::Int64, st::Int64)
    @ Base ./array.jl:724
 [21] collect_to_with_first!(dest::Vector{Symbolics.TermCombination}, v1::Symbolics.TermCombination, itr::Base.Generator{Vector{Any}, Metatheory.Rewriters.PassThrough{Metatheory.Rewriters.Walk{:post, Metatheory.Rewriters.Chain, typeof(Symbolics.basic_simterm), false}}}, st::Int64)
    @ Base ./array.jl:702
 [22] _collect(c::Vector{Any}, itr::Base.Generator{Vector{Any}, Metatheory.Rewriters.PassThrough{Metatheory.Rewriters.Walk{:post, Metatheory.Rewriters.Chain, typeof(Symbolics.basic_simterm), false}}}, #unused#::Base.EltypeUnknown, isz::Base.HasShape{1})
    @ Base ./array.jl:696
 [23] collect_similar
    @ ./array.jl:606 [inlined]
 [24] map(f::Metatheory.Rewriters.PassThrough{Metatheory.Rewriters.Walk{:post, Metatheory.Rewriters.Chain, typeof(Symbolics.basic_simterm), false}}, A::Vector{Any})
    @ Base ./abstractarray.jl:2294
 [25] (::Metatheory.Rewriters.Walk{:post, Metatheory.Rewriters.Chain, typeof(Symbolics.basic_simterm), false})(x::SymbolicUtils.Term{Any, Nothing})
    @ Metatheory.Rewriters ~/.julia/packages/Metatheory/XcKKW/src/Rewriters.jl:198
 [26] (::Metatheory.Rewriters.Fixpoint{Metatheory.Rewriters.Walk{:post, Metatheory.Rewriters.Chain, typeof(Symbolics.basic_simterm), false}})(x::SymbolicUtils.Term{Any, Nothing})
    @ Metatheory.Rewriters ~/.julia/packages/Metatheory/XcKKW/src/Rewriters.jl:118
 [27] hessian_sparsity(f::SymbolicUtils.Mul{Real, Int64, Dict{Any, Number}, Nothing}, u::Vector{SymbolicUtils.Term{Real, Nothing}})
    @ Symbolics ~/.julia/dev/Symbolics/src/diff.jl:576
 [28] sparsehessian(O::Symbolics.Num, vars::Vector{Symbolics.Num}; simplify::Bool)
    @ Symbolics ~/.julia/dev/Symbolics/src/diff.jl:608
 [29] sparsehessian(O::Symbolics.Num, vars::Vector{Symbolics.Num})
    @ Symbolics ~/.julia/dev/Symbolics/src/diff.jl:606
 [30] top-level scope
    @ REPL[10]:1

┆Issue is synchronized with this Trello card by Unito
@baggepinnen
Copy link
Collaborator

I just ran into the same issue, the problem only appears when there are variables in the expression that are not taken derivatives with respect to, e.g.,

julia> @variables a[1:2]
1-element Vector{Symbolics.Arr{Num, 1}}:
 a[1:2]

julia> a = collect(a)
2-element Vector{Num}:
 a[1]
 a[2]

julia> ex = (a[1]+a[2])^2
(a[1] + a[2])^2

julia> Symbolics.hessian(ex, [a[1]])  # non-sparse hessian works
1×1 Matrix{Num}:
 2

julia> Symbolics.sparsehessian(ex, [a[1]])  # Include only one variable => error
ERROR: Failed to apply rule (~f)(~x, ~y) => begin
        #= /home/fredrikb/.julia/packages/Symbolics/1OrKJ/src/diff.jl:556 =#
        if haslinearity_2(~f)
            #= /home/fredrikb/.julia/packages/Symbolics/1OrKJ/src/diff.jl:557 =#
   


julia> Symbolics.sparsehessian(ex, a) # Include both a => no error
2×2 SparseArrays.SparseMatrixCSC{Num, Int64} with 4 stored entries:
 2  2
 2  2

@PeterMaxwell PeterMaxwell mentioned this issue Jul 8, 2022
Open
@devmotion devmotion mentioned this issue Dec 12, 2024
Merged
devmotion added a commit to devmotion/Symbolics.jl that referenced this issue Dec 12, 2024
@devmotion
Add test of JuliaSymbolics#555
72b39f6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Assume unknown functions are non-linear in hessian_sparsity devmotion/Symbolics.jl
2 participants
@baggepinnen @odow