Include test targets in recreate_user_schemes #139

mkantor · 2014-03-17T23:19:10Z

When opening a project with tests in XCode the autogenerated schemes have <TestableReference>s defined in their <Testables> element, but schemes generated by recreate_user_schemes do not, which means xcodebuild test doesn't know what to do.

I think there's some way I can add <TestableReference>s manually (via XCScheme.add_test_target), but if recreate_user_schemes is supposed to act just like XCode then this shouldn't be necessary.

Demonstration

Create a new project in XCode which has at least one test.
Quit XCode.

Run a script like the following:

#!/usr/bin/env bash

PROJECT=TestProject
PROJECT_FILE=$PROJECT.xcodeproj

# Clean up generated schemes.
rm -rf $PROJECT_FILE/xcuserdata/*.xcuserdatad/xcschemes

# Get XCode to generate schemes.
open "$PROJECT_FILE"
sleep 5

xcodebuild test -project "$PROJECT_FILE" -scheme "$PROJECT" -sdk iphonesimulator

It should work and the test(s) should run.

Quit XCode.

Now for the bug. Run a script like the following:

#!/usr/bin/env bash

PROJECT=TestProject
PROJECT_FILE=$PROJECT.xcodeproj

# Clean up generated schemes.
rm -rf $PROJECT_FILE/xcuserdata/*.xcuserdatad/xcschemes

# Use Xcodeproj to generate schemes.
gem install xcodeproj
echo "
 require 'xcodeproj'
 project = Xcodeproj::Project.open '$PROJECT_FILE'
 project.recreate_user_schemes
 project.save
" | ruby

xcodebuild test -project "$PROJECT_FILE" -scheme "$PROJECT" -sdk iphonesimulator

xcodebuild will complain about "Scheme TestProject is not currently configured for the test action", and if you inspect the generated scheme you'll notice that it doesn't specify any <TestableReference>s.

The text was updated successfully, but these errors were encountered:

mkantor · 2014-03-17T23:19:38Z

I came across this while using XCode 5.1, but I'm not sure if it's only that version which has problems.

mkantor · 2014-03-18T00:07:01Z

If there is in fact a straightforward way to add test targets manually, I'd love to hear it. After a few attempts I couldn't figure out how to make it work so for now I'm giving in and adding shared schemes to my project.

fabiopelosin · 2014-03-18T07:32:44Z

Can you post the diff of the files after being edited by xcodeproj?

mkantor · 2014-03-18T17:08:45Z

Here's a diff of xcschemes directories generated by each method (with XCode files first):

diff -Nur xcode-xcschemes/TestProject.xcscheme xcodeproj-xcschemes/TestProject.xcscheme
--- xcode-xcschemes/TestProject.xcscheme    2014-03-18 09:49:54.000000000 -0700
+++ xcodeproj-xcschemes/TestProject.xcscheme    2014-03-18 09:56:56.000000000 -0700
@@ -1,6 +1,6 @@
 <?xml version="1.0" encoding="UTF-8"?>
 <Scheme
-   LastUpgradeVersion = "0510"
+   LastUpgradeVersion = "0500"
    version = "1.3">
    <BuildAction
       parallelizeBuildables = "YES"
@@ -28,26 +28,7 @@
       shouldUseLaunchSchemeArgsEnv = "YES"
       buildConfiguration = "Debug">
       <Testables>
-         <TestableReference
-            skipped = "NO">
-            <BuildableReference
-               BuildableIdentifier = "primary"
-               BlueprintIdentifier = "A2CDAA1618D8AFB500C90658"
-               BuildableName = "TestProjectTests.xctest"
-               BlueprintName = "TestProjectTests"
-               ReferencedContainer = "container:TestProject.xcodeproj">
-            </BuildableReference>
-         </TestableReference>
       </Testables>
-      <MacroExpansion>
-         <BuildableReference
-            BuildableIdentifier = "primary"
-            BlueprintIdentifier = "A2CDA9FB18D8AFB500C90658"
-            BuildableName = "TestProject.app"
-            BlueprintName = "TestProject"
-            ReferencedContainer = "container:TestProject.xcodeproj">
-         </BuildableReference>
-      </MacroExpansion>
    </TestAction>
    <LaunchAction
       selectedDebuggerIdentifier = "Xcode.DebuggerFoundation.Debugger.LLDB"
@@ -58,15 +39,6 @@
       ignoresPersistentStateOnLaunch = "NO"
       debugDocumentVersioning = "YES"
       allowLocationSimulation = "YES">
-      <BuildableProductRunnable>
-         <BuildableReference
-            BuildableIdentifier = "primary"
-            BlueprintIdentifier = "A2CDA9FB18D8AFB500C90658"
-            BuildableName = "TestProject.app"
-            BlueprintName = "TestProject"
-            ReferencedContainer = "container:TestProject.xcodeproj">
-         </BuildableReference>
-      </BuildableProductRunnable>
       <AdditionalOptions>
       </AdditionalOptions>
    </LaunchAction>
@@ -76,15 +48,6 @@
       useCustomWorkingDirectory = "NO"
       buildConfiguration = "Release"
       debugDocumentVersioning = "YES">
-      <BuildableProductRunnable>
-         <BuildableReference
-            BuildableIdentifier = "primary"
-            BlueprintIdentifier = "A2CDA9FB18D8AFB500C90658"
-            BuildableName = "TestProject.app"
-            BlueprintName = "TestProject"
-            ReferencedContainer = "container:TestProject.xcodeproj">
-         </BuildableReference>
-      </BuildableProductRunnable>
    </ProfileAction>
    <AnalyzeAction
       buildConfiguration = "Debug">
diff -Nur xcode-xcschemes/TestProjectTests.xcscheme xcodeproj-xcschemes/TestProjectTests.xcscheme
--- xcode-xcschemes/TestProjectTests.xcscheme   1969-12-31 16:00:00.000000000 -0800
+++ xcodeproj-xcschemes/TestProjectTests.xcscheme   2014-03-18 09:56:56.000000000 -0700
@@ -0,0 +1,59 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<Scheme
+   LastUpgradeVersion = "0500"
+   version = "1.3">
+   <BuildAction
+      parallelizeBuildables = "YES"
+      buildImplicitDependencies = "YES">
+      <BuildActionEntries>
+         <BuildActionEntry
+            buildForTesting = "YES"
+            buildForRunning = "YES"
+            buildForProfiling = "YES"
+            buildForArchiving = "YES"
+            buildForAnalyzing = "YES">
+            <BuildableReference
+               BuildableIdentifier = "primary"
+               BlueprintIdentifier = "A2CDAA1618D8AFB500C90658"
+               BuildableName = "TestProjectTests.xctest"
+               BlueprintName = "TestProjectTests"
+               ReferencedContainer = "container:TestProject.xcodeproj">
+            </BuildableReference>
+         </BuildActionEntry>
+      </BuildActionEntries>
+   </BuildAction>
+   <TestAction
+      selectedDebuggerIdentifier = "Xcode.DebuggerFoundation.Debugger.LLDB"
+      selectedLauncherIdentifier = "Xcode.DebuggerFoundation.Launcher.LLDB"
+      shouldUseLaunchSchemeArgsEnv = "YES"
+      buildConfiguration = "Debug">
+      <Testables>
+      </Testables>
+   </TestAction>
+   <LaunchAction
+      selectedDebuggerIdentifier = "Xcode.DebuggerFoundation.Debugger.LLDB"
+      selectedLauncherIdentifier = "Xcode.DebuggerFoundation.Launcher.LLDB"
+      launchStyle = "0"
+      useCustomWorkingDirectory = "NO"
+      buildConfiguration = "Debug"
+      ignoresPersistentStateOnLaunch = "NO"
+      debugDocumentVersioning = "YES"
+      allowLocationSimulation = "YES">
+      <AdditionalOptions>
+      </AdditionalOptions>
+   </LaunchAction>
+   <ProfileAction
+      shouldUseLaunchSchemeArgsEnv = "YES"
+      savedToolIdentifier = ""
+      useCustomWorkingDirectory = "NO"
+      buildConfiguration = "Release"
+      debugDocumentVersioning = "YES">
+   </ProfileAction>
+   <AnalyzeAction
+      buildConfiguration = "Debug">
+   </AnalyzeAction>
+   <ArchiveAction
+      buildConfiguration = "Release"
+      revealArchiveInOrganizer = "YES">
+   </ArchiveAction>
+</Scheme>
diff -Nur xcode-xcschemes/xcschememanagement.plist xcodeproj-xcschemes/xcschememanagement.plist
--- xcode-xcschemes/xcschememanagement.plist    2014-03-18 09:49:54.000000000 -0700
+++ xcodeproj-xcschemes/xcschememanagement.plist    2014-03-18 09:56:56.000000000 -0700
@@ -6,22 +6,16 @@
    <dict>
        <key>TestProject.xcscheme</key>
        <dict>
-           <key>orderHint</key>
-           <integer>0</integer>
-       </dict>
-   </dict>
-   <key>SuppressBuildableAutocreation</key>
-   <dict>
-       <key>A2CDA9FB18D8AFB500C90658</key>
-       <dict>
-           <key>primary</key>
+           <key>isShown</key>
            <true/>
        </dict>
-       <key>A2CDAA1618D8AFB500C90658</key>
+       <key>TestProjectTests.xcscheme</key>
        <dict>
-           <key>primary</key>
+           <key>isShown</key>
            <true/>
        </dict>
    </dict>
+   <key>SuppressBuildableAutocreation</key>
+   <dict/>
 </dict>
 </plist>

Note that only Xcodeproj generates the TestProjectTests scheme.

fabiopelosin · 2014-03-27T07:34:18Z

I think that the only missing piece to implement this is understanding the logic which Xcode uses to identify schemes? Is is based on the naming convention Target & TargetTestes?

peetasan · 2014-04-14T15:27:42Z

I had a similar issue and my diff (for only one of my targets) is below. Basically instead of a BuildActionEntry there is a TestableReference, the latter after opening the project with Xcode

diff --git a/project-a-unittest.xcscheme b/project-a-unittest 2.xcscheme
index 692535e..644ec99 100644
--- a/project-a-unittest.xcscheme
+++ b/project-a-unittest 2.xcscheme     
@@ -5,22 +5,6 @@
    <BuildAction
       parallelizeBuildables = "YES"
       buildImplicitDependencies = "YES">
-      <BuildActionEntries>
-         <BuildActionEntry
-            buildForTesting = "YES"
-            buildForRunning = "YES"
-            buildForProfiling = "YES"
-            buildForArchiving = "YES"
-            buildForAnalyzing = "YES">
-            <BuildableReference
-               BuildableIdentifier = "primary"
-               BlueprintIdentifier = "7C7124886C864E99BAAC79B5"
-               BuildableName = "project-a-unittest"
-               BlueprintName = "project-a-unittest"
-               ReferencedContainer = "container:ios_x86/xcode/project-a.xcodeproj">
-            </BuildableReference>
-         </BuildActionEntry>
-      </BuildActionEntries>
    </BuildAction>
    <TestAction
       selectedDebuggerIdentifier = "Xcode.DebuggerFoundation.Debugger.LLDB"
@@ -28,6 +12,16 @@
       shouldUseLaunchSchemeArgsEnv = "YES"
       buildConfiguration = "Debug">
       <Testables>
+         <TestableReference
+            skipped = "NO">
+            <BuildableReference
+               BuildableIdentifier = "primary"
+               BlueprintIdentifier = "7C7124886C864E99BAAC79B5"
+               BuildableName = "project-a-unittest.xctest"
+               BlueprintName = "project-a-unittest"
+               ReferencedContainer = "container:build/ios_x86/xcode/project-a.xcodeproj">
+            </BuildableReference>
+         </TestableReference>
       </Testables>
    </TestAction>
    <LaunchAction

kylef · 2015-01-31T22:29:48Z

Dependant on #227.

tkohub · 2015-05-22T16:48:37Z

Does anyone have a workaround to get this running ?

I want to set up a test target and run it like this:

require 'rubygems'
require 'xcodeproj'

project = Xcodeproj::Project.new('test.xcodeproj')

test_target = project.new_target(:unit_test_bundle, 'testing', :ios, '8.0',project.main_group,:objc)
test_target.add_system_frameworks(['XCTest', 'UIKit', 'Foundation'])

// Add a file to the project in the main group
file_name = 'test.m'
file = project.new_file(file_name, :group)

// Add the file to the main target
test_target.add_file_references([file])

//project.recreate_user_schemes

project.save()

Option 1
If i am not using the recreate_user_schemes line i have to open xcode first and then xcodebuild -scheme testing -target testing test works. (Seems like xcode repairs the project somehow because if i run the command directly i get 'xcodebuild: error: The project 'test' does not contain a scheme named 'testing'.'

Option 2
If i am using the recreate_user_schemes i get 'xcodebuild: error: Scheme testing is not currently configured for the test action.' I guess this is related to the bug described aboce

Both options have problems, Option 1 does not work automatically the second one does not work at all.

Do you have any idea how i can get this running ?

AliSoftware · 2015-08-09T12:15:14Z

@tkohub I just merged #288 which now allows you to manipulate XCScheme directly and easily add TestableReferences and all. It should help you with your issue and allow you to configure your scheme and targets to be tested anyhow you want :-) (and hopefully make a PR to add it to recreate_user_schemes 😉

(Note: #288 is on master but hasn't been released yet — it will be in the next rubygem release —so you'll need to either use Xcodeproj's master or wait for the next Xcodeproj release)

Keep us posted to tell if it helped solved your needs.

fabiopelosin added the feature label Mar 27, 2014

fabiopelosin changed the title ~~Schemes generated by recreate_user_schemes do not specify test targets~~ Include test targets in recreate_user_schemes Mar 27, 2014

fabiopelosin added d2:moderate and removed s2:confirmed labels Jul 22, 2014

tkohub mentioned this issue May 26, 2015

Support Build Settings of Test Targets #227

Open

1ec5 mentioned this issue Dec 24, 2015

OS X SDK unit tests mapbox/mapbox-gl-native#3406

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Include test targets in recreate_user_schemes #139

Include test targets in recreate_user_schemes #139

mkantor commented Mar 17, 2014

mkantor commented Mar 17, 2014

mkantor commented Mar 18, 2014

fabiopelosin commented Mar 18, 2014

mkantor commented Mar 18, 2014

fabiopelosin commented Mar 27, 2014

peetasan commented Apr 14, 2014

kylef commented Jan 31, 2015

tkohub commented May 22, 2015

AliSoftware commented Aug 9, 2015

Include test targets in recreate_user_schemes #139

Include test targets in recreate_user_schemes #139

Comments

mkantor commented Mar 17, 2014

Demonstration

mkantor commented Mar 17, 2014

mkantor commented Mar 18, 2014

fabiopelosin commented Mar 18, 2014

mkantor commented Mar 18, 2014

fabiopelosin commented Mar 27, 2014

peetasan commented Apr 14, 2014

kylef commented Jan 31, 2015

tkohub commented May 22, 2015

AliSoftware commented Aug 9, 2015