Assertion error while analyzing XTA call graph for hsqldb 2.0.0

[svenkeidel]

Hi,

Thanks for your hard work on Opal.

I encountered an assertion error while computing the XTA call graph for HSQLDB 2.0.0 (log.txt). I encountered this error with this version of Opal (https://github.com/opalj/opal/tree/fix/scala-reflect-issue-with-github-pureconfig) based of the release 7.0.0. As analysis configuration I used the updated configuration in the JCG Opal adapter (https://github.com/svenkeidel/JCG/blob/develop/jcg_opal_testadapter/src/main/scala/OpalJCGAdatper.scala).

Here are the jars for you to reproduce hsqldb-2.0.0-jars.tar.gz.

Regards,
Sven

[svenkeidel]

The stack trace says that the second assertion failed.

final class TypePropagationAnalysis private[analyses] (
    val project:         SomeProject,
    selectTypeSetEntity: TypeSetEntitySelector
) extends ReachableMethodAnalysis {
 ...
 private def processCallees(
        callees: Callees
    )(
        implicit
        state:          State,
        partialResults: ArrayBuffer[SomePartialResult]
    ): Unit = {
        val bytecodeOpt = if (state.methodHasBody) Some(state.callContext.method.definedMethod.body.get) else None
        for {
            pc <- callees.callSitePCs(state.callContext)
            calleeContext <- callees.callees(state.callContext, pc)
            callee = calleeContext.method
            if !state.isSeenCallee(pc, callee) && !isIgnoredCallee(callee)
        } {
            // Some sanity checks ...
            // Methods with multiple defined methods should never appear as callees.
            elidedAssert(!callee.hasMultipleDefinedMethods)
            // Instances of DefinedMethod we see should only be those where the method is defined in the class file of
            // the declaring class type (i.e., it is not a DefinedMethod instance of some inherited method).  However,
            // in inconsistent bytecode scenarios, the call graph may resolve to a non-implemented abstract method.
            elidedAssert(!callee.hasSingleDefinedMethod ||
                (callee.declaringClassType == callee.asDefinedMethod.definedMethod.classFile.thisType) ||
                callee.asDefinedMethod.definedMethod.isAbstract)

            ...
        }
    }
    ...
}

[svenkeidel]

The only way this condition becomes false is when

callee.hasSingleDefinedMethod == True
callee.declaringClassType != callee.asDefinedMethod.definedMethod.classFile.thisType
callee.asDefinedMethod.definedMethod.isAbstract == False

[errt]

Thanks for reporting. I did have a look into this and could reproduce the behavior. The call graph actually contains edges to methods where the declaring class type differs from the type defining the method. This should be unproblematic here (the assertion has been added long time ago and it is unclear what is should actually guard against), but may still be unintended.

I digged deeper into one case and found this: The (package private) caller method contained a call to a private method. The call graph tried to resolve this call for all possible types of this, which include subtypes. It does so in CallGraphAnalysis:90 by using ProjectLike.instanceCall, which in turn uses ProjectLike.lookupVirtualMethod that returns the (or rather, a intermediate subclass', but that shouldn't matter) private method, which ProjectLike.instanceCall identifies as non-accessible by the supposed caller. It does return an empty results, prompting CallGraphAnalysis to treat this as an unknown target that would usually be resolved to a VirtualDeclaredMethod. However, since the method exists, despite being private, the call to DeclaredMethods.apply in CallGraphAnalysis:358 returns the private method with the observed differing declaring class type and type defining the method.

I suspect this happens because the JVM has changed from using invokespecial to using invokevirtual to invoke private methods, but our call graph analysis does not treat this correctly and tries to resolve this call as if it was targeting an actual virtual method. We will probably have to dig into the exact behavior the JVM has when invoking private methods using invokevirtual to fix this.

As a workaround, it should be fine to just remove this assertion, or, maybe better, to not process any callees where the declaring class type and type defining the method differ.

[svenkeidel]

Thank you very much for the deep dive. I'll remove the assertion for now to fix the error for now and try again.