<![CDATA[A software Author's Website]]> 2014-01-02T12:33:26+00:00 http://graoil.co.uk/ Octopress <![CDATA[Using Catch for Jeff Langr's TDD book chapter 2]]> 2013-12-30T16:12:00+00:00 http://graoil.co.uk/blog/2013/12/30/using-catch-for-jeff-langrs-tdd-book-chapter-2 As I'm a bit of a fan of Phil Nash's Catch test framework I decided to convert the code in Jeff Langr's book to use Catch. This post describes the changes that were required for the first tests, i.e. chapter 2.

Code for this project is here…

Basic changes

Catch is header only so is much simpler to incorporate than Google Mock/Test so the CMakeLists.txt can be much simpler. For the start of the chapter, this is sufficient:

1
2
3
4
5
6
7
8
9
10
11
12

project(chapterFirstExample)
cmake_minimum_required(VERSION 2.6)

include_directories(../../Catch/include)
add_definitions(-std=c++0x)
set(CMAKE_CXX_FLAGS "${CMAXE_CXX_FLAGS} -Wall")

set(sources
   main.cpp
   SoundexTest.cpp)
add_executable(test ${sources})
target_link_libraries(test pthread)

All we need to do is change the path to the Catch include directory, remove the link path and remove the link libraries.

Similarly, main.cpp is also simpler:

1
2

#define CATCH_CONFIG_MAIN
#include "catch.hpp"

Strictly speaking we don't need main.cpp at all, but it simplifies the test code to keep it.

First 4 examples

The first examples are not very challenging and all that needs to be changed is to #include "catch.hpp" and change the TEST macro for a TEST_CASE with quotes around the identifiers, e.g. example 4:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

#include <string>

class Soundex
{
public:
   std::string encode(const std::string& word) const {
      return "";
   }
};

#include "catch.hpp"

TEST_CASE("SoundexEncoding", "RetainsSoleLetterOfOneLetterWord") {
   Soundex soundex;

   auto encoded = soundex.encode("A");
}

Examples 5-7

Catch provides a much richer assertion mechanism so we can replace the ASSERT_THAT macro with a REQUIRE macro:

1
2
3
4
5
6
7
8

//...
TEST_CASE("SoundexEncoding", "RetainsSoleLetterOfOneLetterWord") {
   Soundex soundex;

   auto encoded = soundex.encode("A");

   REQUIRE(encoded == "A");
}
Running this for example 5 gives the following output in Catch: 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

test.exe is a Catch v1.0 b23 host application.
Run with -? for options

-------------------------------------------------------------------------------
SoundexEncoding
-------------------------------------------------------------------------------
c:/Projects/git/jlangr/c2/SoundexTest.cpp:18
...............................................................................

c:/Projects/git/jlangr/c2/SoundexTest.cpp:24: FAILED:
  REQUIRE( encoded == "A" )
with expansion:
  "" == "A"

===============================================================================
1 test case - failed (1 assertion - failed)
As you can see, the output provides both the original expression and the reconstructed expression with the values the caused the assertion failure. Much better - Jeff’s explanation would have been much shorter with this! The fix in example 6 gives this output: 
1

All tests passed (1 assertion in 1 test case)

In Catch, example 7 id identical to example 6 so has been removed.

Examples 9

This requires the same type of changes as above, except that the test case name needs to be unique…we'll fix that in example 10.

Example 10,11

We can use Catch SECTIONs to simplify this example…so we don't need the SoundexEncoding class at all, and the test looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

TEST_CASE("SoundexEncoding", "RetainsSoleLetterOfOneLetterWord") {
   Soundex soundex;

   SECTION("RetainsSoleLetterOfOneLetterWord") {
      auto encoded = soundex.encode("A");

      REQUIRE(encoded == "A000");
   }

   SECTION("PadsWithZerosToEnsureThreeDigits") {
      auto encoded = soundex.encode("I");

      REQUIRE(encoded == "I000");
   }
}

Catch will run the TEST_CASE for each SECTION, automatically initialising the soundex object on each run.

Example 11 is similar.

Examples 12-17

Example 12 requires similar changes, but fails (as expected):

1
2
3
4
5
6
7
8
9
10
11

-------------------------------------------------------------------------------
SoundexEncoding
  ReplacesConsonantsWithAppropriateDigits
-------------------------------------------------------------------------------
c:/Projects/git/jlangr/c2/SoundexTest.cpp:7
...............................................................................

c:/Projects/git/jlangr/c2/SoundexTest.cpp:19: FAILED:
  REQUIRE( soundex.encode("Ab") == "A100" )
with expansion:
  "Ab000" == "A100"

Again, the output from Catch is much more helpful for diagnosing the error, providing the TEST_CASE name, the SECTION and the expansions.

Examples 13-17 follow the same pattern.

Examples 18-22

Examples 18-22 are similar but we can use the Catch CHECK() macro instead of EXPECT_THAT(), e.g.:

1
2
3
4
5

   //...
   SECTION("ReplacesConsonantsWithAppropriateDigits") {
      CHECK(soundex.encode("Ab") == "A100");
      CHECK(soundex.encode("Ac") == "A200");
   }

Examples 23-25

In example 23, Jeff disables a test. We can't do the same thing with a Catch SECTION (yet…) but we can disable an entire test by using the '[hide]' tag, so we temporarily put the test in a TEST_CASE on its own:

1
2
3
4

TEST_CASE("ReplacesMultipleConsonantsWithDigits", "[hide]") {
   Soundex soundex;
   REQUIRE(soundex.encode("Acdl") == "A234");
}

Example 26-31

In passing, note that Catch also detects the exception:

1
2
3
4
5
6
7
8
9
10

SoundexEncoding
  LimitsLengthToFourCharacters
-------------------------------------------------------------------------------
c:/Projects/git/jlangr/c2/SoundexTest.cpp:5
...............................................................................

c:/Projects/git/jlangr/c2/SoundexTest.cpp:25: FAILED:
  REQUIRE( soundex.encode("Dcdlb").length() == 4u )
due to unexpected exception with message:
  basic_string::_S_create

Example 32

To use StartsWith in Catch, we must use the REQUIRE_THAT() macro, e.g.:

1
2
3

   SECTION("UppercasesFirstLetter") {
      REQUIRE_THAT(soundex.encode("abcd"), StartsWith("A"));
   }

Examples 33-39

These all follow the same pattern as above.

]]> <![CDATA[Integrating Phil Nash's Catch C++ Unit Test Framework into Visual Studio]]> 2013-11-17T19:44:00+00:00 http://graoil.co.uk/blog/2013/11/17/integrating-phil-nashs-catch-c-plus-plus-unit-test-framework-into-visual-studio After I adapted some existing code that used MSTest so that I was able to use Catch, I started to look at whether it was possible to do better; specifically I wanted to share source code between VS and Catch written for a console app and I wanted to retain the ability to generate standard VS TestResults files (*.trx).

The results were better than I expected; I found it was possible, with a little adaptation, to change the Catch internals so that I could do all that; so much so that I was able to reimplement the Catch self tests as Managed C++ test projects for VS2010 and VS2012 as well as Native C++ test projects.

You can find a fork of Catch here that implements this; hopefully Phil and I will be able to integrate this into the Catch mainline soon but until that is done I will do my best to keep the fork in sync with the code here.

You can find details on how to use it here and it looks like this:

VS2010:

VS2010

VS2012:

VS2012

]]> <![CDATA[Replacing MSTest with Phil Nash's Catch framework for VS2012 native C++ tests]]> 2013-10-29T19:31:00+00:00 http://graoil.co.uk/blog/2013/10/29/using-phil-nashs-catch-framework-for-VS2012-native-tests In an earlier post I described how to share the same codebase for MSTest managed C++ code and Phil Nash's Catch framework. In VS2012, there's an option for 'Native' C++ tests; this article describes how to use Catch to share the codebase for Native tests too.

VS2012 and Native C++ Tests

In VS2012, you can create a managed C++ test project like this:

Managed C++

You can also create a Native test, like this:

Native C++

In my case, a native test looks something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

namespace NativeTestProject
{      
  TEST_CLASS(UnitTest1)
  {
        NativeTestClass* m_data;
  public:
      UnitTest1()
        {}
        ~UnitTest1()
        {
        }
        TEST_METHOD_INITIALIZE(TestInit)
        {
            m_data = new NativeTestClass("Fred", 42);
        }
        TEST_METHOD_CLEANUP(TestCleanup)
        {
            delete m_data;
            m_data = NULL;
        }

      TEST_METHOD(TestMethod1)
      {
            ...
        }

I'm not using the constructor/destructor for initialisation and cleanup; instead I'm using the TEST_METHOD_INITIALIZE/CLEANUP macros that call setup and cleanup code before each method is called. This is very similar to my previous article…in fact it's so similar that all I need to do is redefine my macro for MS_TEST_CASE_METHOD and define REQUIRE/FAIL and I'm done!

We need a different test runner for native tests:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

C:\Projects\NativeTestProject>"C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\CommonExtensions\Microsof
t\TestWindow\vstest.console.exe" /Logger:Trx debug\NativeTestProject.dll
Microsoft (R) Test Execution Command Line Tool Version 11.0.60315.1
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Passed   Method1
Failed   Method2
Error Message:
   Assert failed
Stack Trace:
        at ManagedTestProject::TestWithoutInitializeAndCleanup::Method2() in c:\projects\nativetestproject\unittestcatch
.cpp:line 23
Passed   Method1
Failed   Method2
Error Message:
   Assert failed
Stack Trace:
        at ManagedTestProject::TestWithInitializeAndCleanup::Method2() in c:\projects\nativetestproject\unittestcatch.cp
p:line 53
Results File: C:\Projects\NativeTestProject\TestResults\name_machine 2013-10-28 20_59_53.trx

Total tests: 4. Passed: 2. Failed: 2. Skipped: 0.
Test Run Failed.
Test execution time: 0.8892 Seconds

The code is identical to the managed/Catch example, so Catch works just the same:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

C:\Projects\NativeTestProject\Debug>CatchTestProject.exe

CatchTestProject.exe is a Catch v1.0 b11 host application.
Run with -? for options

-------------------------------------------------------------------------------
TestWithoutInitializeAndCleanup::Method2
-------------------------------------------------------------------------------
c:\projects\nativetestproject\unittestcatch.cpp(18)
...............................................................................

c:\projects\nativetestproject\unittestcatch.cpp(23): FAILED:
  REQUIRE( result != "Fred" )
with expansion:
  "Fred" != "Fred"

-------------------------------------------------------------------------------
TestWithInitializeAndCleanup::Method2
-------------------------------------------------------------------------------
c:\projects\nativetestproject\unittestcatch.cpp(49)
...............................................................................

c:\projects\nativetestproject\unittestcatch.cpp(53): FAILED:
  REQUIRE( result != "Fred" )
with expansion:
  "Fred" != "Fred"

===============================================================================
4 test cases - 2 failed (8 assertions - 2 failed)

Project used in this article:

NativeTestProject NativeTestLibrary

]]> <![CDATA[Replacing MSTest with Phil Nash's Catch framework for managed C++ tests]]> 2013-10-28T19:47:00+00:00 http://graoil.co.uk/blog/2013/10/28/replacing-mstest-with-phil-nashs-catch-framework-for-managed-tests Visual Studio 2010 has an option for creating C++ unit tests, but these tests are 'managed' C++. There are a number of problems with such tests:

  1. The tests are slow to compile and slow to load/run.
  2. Tests tend to be more complicated than needed - the Managed C++ compiler does not allow C++ value types as member variables for example, so these have to be kept as pointers and new/delete used.
  3. The debugger doesn't understand C++ types very well, so when in managed code it is often not possible to view object values. Whilst it could be argued that using TDD should avoid most uses of the debugger, sometimes it's inevitable.
  4. Sometimes the Managed C++ debugger gets a 'mind of its own' and decides to run a test to completion part way through debugging. This can be frustrating!
  5. Sometimes, the process vstest.executionengine.x86.exe gets left running after the tests have completed whcih prevents the any new builds from compiling as the DLL is held open by the executionengine. There is a bug report for this but apparently no solution other than using taskkill as a pre-build event (http://stackoverflow.com/questions/13497168/vstest-executionengine-x86-exe-not-closing)

In the absence of Native C++ unit tests (note: VS2012 has these - see later) I wanted to be able to share the code base between MSTest managed tests and native tests so that the build system could happily run MSTest on its managed build and I could run native tests using Catch. This article describes my experiment to do this…

Differences in test philosophy

To persuade MSTest to look like Catch, we have to think about how the test needs to run. MSTest requires a class, but Catch does not. Catch allows multiple runs by using SECTIONs, but this doesn't figure in MSTest. So my first attempt used a TEST_CASE as a 'class' wrapper and a SECTION for each test method. This worked pretty well, except that it wasn't possible to specify individual tests to run (SECTIONs don't support tags).

Looking at the tests that I wanted convert, they tended to follow a pattern:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

namespace ManagedTestProject
{
    [TestClass]
    public ref class ClassNameForTest
    {
        // some data that needs to be initialized...
        Stuff* m_data;

      //Use TestInitialize to run code before running each test
      [TestInitialize()]
      void MyTestInitialize()
        {
            m_data = new Stuff(...);
        }

      //Use TestCleanup to run code after each test has run
      [TestCleanup()]
      void MyTestCleanup()
        {
            delete m_data;
            m_data = NULL;
        }

    public:
      [TestMethod]
        void Method1() {}

      [TestMethod]
        void Method2() {}
    };
}

A specific solution

Fortunately, they always did this; declare pointer member variables for objects the test needed, create in TestInitialize and destroy in TestCleanup, in that order, followed by test methods. I realised that I could declare the data at namespace scope and wrap the initialisation/destruction in a helper class, something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

namespace x
{
    Stuff* m_data;
    void setup()
    {
        m_data = new Stuff();
    }
    void cleanup()
    {
        delete m_data;
        m_data = NULL;
    }

    struct setup_wrapper
    {
        setup_wrapper()
        {
            setup();
        }
        virtual ~setup_wrapper()
        {
            cleanup();
        }
    };

    TEST_CASE_METHOD(setup_wrapper, "Method1");
}
TEST_CASE_METHOD is a Catch macro that creates a class instance derived from setup_wrapper for each method, then runs the test in its own method. Unfortunately, not all my tests had TestInitiaize/TestCleanup methods, so I wanted to be able to specify those in a similar way to Native C++ tests. I tried several ways to do this, but ended up with a couple of macros, used like this: 
1
2
3
4
5
6
7
8
9

TEST_METHOD_INITIALIZE(setupMethodName)
{
    m_data = new Stuff(...);
}

TEST_METHOD_CLEANUP(cleanupMethodName)
{
    delete m_data; m_data = NULL;
}
and declared like this: 
1
2
3
4
5

#define TEST_METHOD_INITIALIZE(methodName) \
    void setup(int)

#define TEST_METHOD_CLEANUP(methodName) \
    void cleanup(int)

The wrapper class then calls each:

1
2
3
4
5
6
7
8

    setup_wrapper()
    {
        setup(0);
    }
    virtual ~setup_wrapper()
    {
        cleanup(0);
    }

If there isn't a setup method, there won't be a setup(int) or a cleanup(int) but we can fix that by defining a templated setup/cleanup function at namespace scope:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

namespace X
{
    template <typename T>
    void setup(T t) { }
    template <typename T>
    void cleanup(T t) { }
    ...
    // no TEST_METHOD_INITIALIZE/CLEANUP
    struct setup_wrapper
    {
        setup_wrapper()
        {
            // calls template function
            setup(0);
        }
        virtual ~setup_wrapper()
        {
            // calls template function
            cleanup(0);
        }
    };

    TEST_CASE_METHOD(setup_wrapper, "Method1");
}
C++ overload resolution will always prefer the non-templated `setup(int)` if it exists, so will call the non-templated method. # A final solution # Now I can embed all the nasty bits in macros and mangle some names to avoid duplicate definitions so that my final shared codebase looks like this: 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61

#include "stdafx.h"
#include "MSManagedTestMacros.hpp"

#include "lib.h"    // code under test, declares NativeTestClass

namespace ManagedTestProject {

    TEST_CLASS(TestWithoutInitializeAndCleanup)
    {
        TEST_CLASS_CONTEXT()

        MS_TEST_CASE_METHOD(Method1)
        {
            NativeTestClass* m_data = new NativeTestClass("Fred", 42);
            REQUIRE(m_data != NULL);
            std::string result = m_data->getName();
            REQUIRE(result == "Fred");
        }

        MS_TEST_CASE_METHOD(Method2)
        {
            NativeTestClass* m_data = new NativeTestClass("Fred", 42);
            REQUIRE(m_data != NULL);
            std::string result = m_data->getName();
            REQUIRE(result != "Fred");
        }
    };

    TEST_CLASS(TestWithInitializeAndCleanup)
    {
        TEST_CLASS_CONTEXT()

        NativeTestClass* m_data;

        TEST_METHOD_INITIALIZE(MyInitialize)
        {
            m_data = new NativeTestClass("Fred", 42);
        }
      
        TEST_METHOD_CLEANUP(MyCleanup)
        {
            delete m_data;
            m_data = NULL;
        }

        MS_TEST_CASE_METHOD(Method1)
        {
            REQUIRE(m_data != NULL);
            std::string result = m_data->getName();
            REQUIRE(result == "Fred");
        }

        MS_TEST_CASE_METHOD(Method2)
        {
            REQUIRE(m_data != NULL);
            std::string result = m_data->getName();
            REQUIRE(result != "Fred");
        }
    };

} // namespace

Running this with MSTest gives this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

C:\Projects\ManagedTestProject>"c:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\MSTest.exe" /testcontain
er:Debug\DefaultTest.dll
Microsoft (R) Test Execution Command Line Tool Version 11.0.50727.1
Copyright (c) Microsoft Corporation. All rights reserved.

Loading Debug\DefaultTest.dll...
Starting execution...

Results               Top Level Tests
-------               ---------------
Passed                ManagedTestProject.TestWithInitializeAndCleanup.Method1
Failed                ManagedTestProject.TestWithInitializeAndCleanup.Method2
Passed                ManagedTestProject.TestWithoutInitializeAndCleanup.Method1
Failed                ManagedTestProject.TestWithoutInitializeAndCleanup.Method2
2/4 test(s) Passed, 2 Failed

Summary
-------
Test Run Failed.
  Passed  2
  Failed  2
  ---------
  Total   4
Results file:  C:\Projects\ManagedTestProject\TestResults\name_machine 2013-10-28 20_24_15.trx
Test Settings: Default Test Settings

Recompiling and running the same code with Catch gives:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

C:\Projects\ManagedTestProject\Debug>CatchTestProject.exe

CatchTestProject.exe is a Catch v1.0 b11 host application.
Run with -? for options

-------------------------------------------------------------------------------
TestWithoutInitializeAndCleanup::Method2
-------------------------------------------------------------------------------
c:\projects\managedtestproject\unittestcatch.cpp(20)
...............................................................................

c:\projects\managedtestproject\unittestcatch.cpp(25): FAILED:
  REQUIRE( result != "Fred" )
with expansion:
  "Fred" != "Fred"

-------------------------------------------------------------------------------
TestWithInitializeAndCleanup::Method2
-------------------------------------------------------------------------------
c:\projects\managedtestproject\unittestcatch.cpp(53)
...............................................................................

c:\projects\managedtestproject\unittestcatch.cpp(57): FAILED:
  REQUIRE( result != "Fred" )
with expansion:
  "Fred" != "Fred"

===============================================================================
4 test cases - 2 failed (8 assertions - 2 failed)

Much more satisfactory…

VS2012 and Native C++ Tests

In VS2010, you can create a C++ test project like this:

VS2010

This is always a managed C++ project and the macros above work fine. In VS2012, there are two options….but I think I'll leave that for another post ;-)

Projects used in this article:

ManagedTestProject NativeTestLibrary

]]> <![CDATA[Debugging Phil Nash's CATCH framework with gdb on Mingw32]]> 2013-10-15T19:31:00+01:00 http://graoil.co.uk/blog/2013/10/15/debugging-phil-nashs-catch-framework-with-gdb-on-mingw32 So you have Mingw (or gdb on Linux) and you'd like to use Phil Nash's CATCH testing framework. This article will go through some simple tests that we can construct and how to use it in that environment.

Throughout this document I use a test project that I've setup in C:\Projects\catch_gdb_example - I'll use the Mingw directory /c/Projects/catch_gdb_example to access it.

For testing, I was using the latest CATCH with Mingw32:

1
2

$ uname -a
MINGW32_NT-6.1 machine 1.0.18(0.48/3/2) 2012-11-21 22:34 i686 Msys
and Git: 
1
2

$ git --version
git version 1.8.3.msysgit.0

First, CATCH your framework

There are two easy ways to get CATCH, and one that is slightly harder, so guess which one I chose? The easy way is download the source zip file, available from https://github.com/philsquared/Catch and unpack it - you can either use the full include headers (from catch/include) or the single header (from catch/single_include).

The alternative that I chose was to first download and install git for windows (http://msysgit.github.com/).

Then I cloned Phil's repository with: 

1
2

cd /c/Projects/catch_gdb_example
git clone https://github.com/philsquared/Catch.git

…this clones the CATCH framework and puts it into the 'Catch' sub–directory.

Build a sanity check

Next create a cpp file (e.g. main.cpp) with the following contents:

1
2
3
4
5
6
7

#define CATCH_CONFIG_MAIN
#include "catch.hpp"

TEST_CASE( "main/sanity check", "[sanity]" )
{
    REQUIRE(true);
}

Now a simple makefile:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

CC=g++
CFLAGS=-c -DDEBUG -I./Catch/include -g -Wall -pedantic -Wextra -std=c++0x
LDFLAGS=
SOURCES= \
    main.cpp
    
OBJECTS=$(SOURCES:.cpp=.o)
EXECUTABLE=test.exe

all: $(SOURCES) $(EXECUTABLE)
  
$(EXECUTABLE): $(OBJECTS) 
  $(CC) $(LDFLAGS) $(OBJECTS) -o $@

.cpp.o:
  $(CC) $(CFLAGS) $< -o $@

clean:
  rm -rf *.o $(EXECUTABLE)

…and we can build the test with: 

1

make

and run with:

1
2
3
4
5
6
7
8

User@machine /c/Projects/catch_gdb_example
$ make
g++ -c -DDEBUG -I./Catch/include -g -Wall -pedantic -Wextra -std=c++0x main.cpp -o main.o
g++  main.o -o test.exe

User@machine /c/Projects/catch_gdb_example
$ ./test.exe main/*
All tests passed (1 assertion in 1 test case)

Now for the debugger

Add the following failing test:

1
2
3
4
5

TEST_CASE("test/fails","[gdb]")
{
    int a = 1;
    REQUIRE(a == 0);
}
and recompile: 
1
2
3
4

User@machine /c/Projects/catch_gdb_example
$ make
g++ -c -DDEBUG -I./Catch/include -g -Wall -pedantic -Wextra -std=c++0x main.cpp -o main.o
g++  main.o -o test.exe

Now you can run the test program and set it to break on the failure of test conditions:

1
2
3
4
5
6
7
8
9
10
11
12

$ gdb ./test
GNU gdb (GDB) 7.6.1
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "mingw32".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from c:\Projects\catch_gdb_example\test.exe...done.
(gdb)

Now you can run the (failing) test case:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

(gdb) r -b
Starting program: c:\Projects\catch_gdb_example/./test.exe -b
[New Thread 8364.0x2398]
 
test.exe is a Catch v1.0 b10 host application.
Run with -? for options

-------------------------------------------------------------------------------
test/fails
-------------------------------------------------------------------------------
main.cpp:9
...............................................................................

main.cpp:12: FAILED:
  REQUIRE( a == 0 )
with expansion:
  1 == 0

 
Program received signal SIGTRAP, Trace/breakpoint trap.
0x7685321a in KERNELBASE!DebugBreak () from C:\windows\syswow64\KernelBase.dll
(gdb)

If you look at the stack you'll see that we aren't actually in the test function:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

0x7685321a in KERNELBASE!DebugBreak () from C:\windows\syswow64\KernelBase.dll
(gdb) bt
#0  0x7685321a in KERNELBASE!DebugBreak () from C:\windows\syswow64\KernelBase.dll
#1  0x0040602d in ____C_A_T_C_H____T_E_S_T____9 () at main.cpp:12
#2  0x0041fcf9 in Catch::FreeFunctionTestCase::invoke (this=0xb72850)
    at ./Catch/include/internal/catch_test_case_registry_impl.hpp:103
#3  0x00403837 in Catch::TestCase::invoke (this=0xb72b10)
    at ./Catch/include/internal/catch_test_case_info.hpp:78
#4  0x00409621 in _fu32___ZSt4cerr () at ./Catch/include/internal/catch_runner_impl.hpp:277
#5  0x0040a066 in Catch::RunContext::runTest (this=0x28fbc8, testCase=...)
    at ./Catch/include/internal/catch_runner_impl.hpp:123
#6  0x00417473 in Catch::Runner::runTestsForGroup (this=0x28fcdc, context=..., filterGroup=...)
    at ./Catch/include/catch_runner.hpp:67
#7  0x00417744 in Catch::Runner::runTests (this=0x28fcdc) at ./Catch/include/catch_runner.hpp:48
#8  0x004182c2 in Catch::Session::run (this=0x28fe6c) at ./Catch/include/catch_runner.hpp:203
#9  0x004181fa in Catch::Session::run (this=0x28fe6c, argc=2, argv=0xb72dc8)
    at ./Catch/include/catch_runner.hpp:186
#10 0x00405959 in main (argc=2, argv=0xb72dc8) at ./Catch/include/internal/catch_default_main.hpp:15
(gdb)

So we need to go up 1 frame and then we can examine variables in the test:

1
2
3
4
5
6

(gdb) up
#1  0x0040602d in ____C_A_T_C_H____T_E_S_T____9 () at main.cpp:12
12          REQUIRE(a == 0);
(gdb) p a
$1 = 1
(gdb)

Oh look, 'a == 1', why did I think it should be 0?

That's all there is to it! Continuing the run gives the expected CATCH output:

1
2
3
4
5
6
7

(gdb) c
Continuing.
===============================================================================
2 test cases - 1 failed (2 assertions - 1 failed)

[Inferior 1 (process 8364) exited with code 01]
(gdb)
]]> <![CDATA[Deconstructing compile time FizzBuzz in C++ without Boost]]> 2013-10-09T05:53:00+01:00 http://graoil.co.uk/blog/2013/10/09/compile-time-fizzbuzz-without-boost Several years ago Adam Peterson published an article on how to implement FizzBuzz at cmpile time in C++. The code was clever, but had a dependency on Boost and didn't go into great detail on how it worked, so I thought I'd write it again from scratch and try to explain my workings. In this article I'm only going to show examples that would work for very small FizzBuzz sequences because, well, this is a learning exercise not a typing exercise!

At the end of this article you'll find the source that I used to run the examples below - up to 16 items in the sequence. Much of what follows is a variation on the code in chapter 5 of C++ Template Metaprogramming, in particular the use of mpl::vector and the 'tiny' sequence.

First, get an error message

To make this work, we first have persuade the compiler to print an error message to the console when we build the program, something like this:

1
2
3
4
5
6
7
8
9
10
11

struct void_;

template <class T0 = void_, class T1 = void_, class T2 = void_, class T3 = void_>
struct vector
{
    typedef vector type;
    typedef T0 T0;
    typedef T1 T1;
    typedef T2 T2;
    typedef T3 T3;
};
This is a simplified version of the mpl::vector example from C++ Template Metaprogramming. In particular note the typedef for ‘self’ and the typedefs for template parameters; we’ll be using these in our helper templates. When we compile this with VS2010, we get this: 
1
2
3
4
5
6
7
8
9
10
11
12

1>ClCompile:
1>  main.cpp
1>c:\projects\fizzbuzz_example\main.cpp(243): error C2039: 'compilation_error_here' : is not a member of 'vector<T0,T1,T2>'
1>          with
1>          [
1>              T0=int,
1>              T1=long,
1>              T2=double
1>          ]
1>c:\projects\fizzbuzz_example\main.cpp(243): error C2146: syntax error : missing ';' before identifier 'res'
1>c:\projects\fizzbuzz_example\main.cpp(243): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
1>c:\projects\fizzbuzz_example\main.cpp(243): error C2065: 'res' : undeclared identifier
This is promising; it displays the types that we put in, and in the right order. That’s fine, but the compiler works with types and we need to display numeric values, so we need a helper to convert intergers to types. [Loki](http://loki-lib.sourceforge.net/ “Loki library”) calls this `Int2Type` and [Boost.MPL](http://www.boost.org/doc/libs/1_54_0/libs/mpl/doc/index.html “Boost.MPL”) has `mpl::int_` that does the same thing, but it’s really easy to implement: 
1
2
3
4
5

template <int N>
struct int_
{
    static const int value = N;
};
For each distinct value ‘N’, this template creates a distinct type, so `int_<0>` is a different type from `int_<1>` etc. Now we can put this into our ‘error’ template: 
1

typedef vector<int_<0>,int_<1>,int_<2> >::compilation_error_here res;
and we get error output with increasing numeric values, which is what we want: 
1
2
3
4
5
6
7

1>c:\projects\fizzbuzz_example\main.cpp(250): error C2039: 'compilation_error_here' : is not a member of 'vector<T0,T1,T2>'
1>          with
1>          [
1>              T0=int_<0>,
1>              T1=int_<1>,
1>              T2=int_<2>
1>          ]

Now generate the sequence…

That's fine, but now we need to generate the sequence.

To get a new sequence, we have to append a new type (e.g. int_<1>) to an existing sequence (int_<0>). Loki uses typelists that we could just append to but here we're modelling our sequence on mpl::vector so we need to do more work.

We can only do that if we know how big our current sequence is, so we first need to specialize our template for every possible size of sequence (this is why we're only working with small sequences!). This is a very much simplified version of the code in C++ Template Metaprogramming, in particular I've not implemented 'apply' metafunctions to make it clearer what's going on. Note that here the size is derived from the int_<> helper template that we defined earlier so that we can get the 'value' later:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

template <class T0, class T1, class T2, class T3>
struct size_impl : int_<4> {};

template <class T0, class T1, class T2>
struct size_impl<T0,T1,T2,void_> : int_<3> {};

template <class T0, class T1>
struct size_impl<T0,T1,void_,void_> : int_<2> {};

template <class T0>
struct size_impl<T0,void_,void_,void_> : int_<1> {};

template <>
struct size_impl<void_,void_,void_,void_> : int_<0> {};

template <class Sequence>
struct size : size_impl<typename Sequence::T0, typename Sequence::T1
    , typename  Sequence::T2, typename  Sequence::T3>
{};

For an existing length, the 'size' template will select the 'best' specialisation of the size_impl template and add the new type as the last parameter. So now we can append a new type to another:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

template <class Sequence, class T, int N>
struct push_back_impl;

template <class Sequence, class T>
struct push_back_impl<Sequence, T, 0>
    : vector<T, void_, void_, void_>
{};

template <class Sequence, class T>
struct push_back_impl<Sequence, T, 1>
    : vector<typename Sequence::T0, T, void_, void_>
{};

template <class Sequence, class T>
struct push_back_impl<Sequence, T, 2>
    : vector<typename Sequence::T0, typename Sequence::T1, T, void_>
{};

template <class Sequence, class T>
struct push_back_impl<Sequence, T, 3>
    : vector<typename Sequence::T0, typename Sequence::T1, typename Sequence::T2, T>
{};

template <class Sequence, class T>
struct push_back
    : push_back_impl<Sequence, T, size<Sequence>::value >
{};

The same idea applies here; we use the push_back template to select the most appropriate push_back_impl template for the existing size. Note that we're not checking that the sequence is already full; this shouldn't be a problem with our limited use case for this example.

Now we can use the recursive parts of Adam's RunFizzBuzz to generate our sequence:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

template<int i>
struct RunFizzBuzz
{
    typedef int_<i> Number;
    typedef typename push_back<typename RunFizzBuzz<i - 1>::type, Number>::type type;
};

template<>
struct RunFizzBuzz<0>
{
    typedef vector<int_<0> > type;
};

int main()
{
    typedef RunFizzBuzz<3>::type::compilation_error_here res;
}

In this case the compiler will use the primary template and will recursively subtract 1 from the number and then instantiate itself until the number gets to zero, when it will use the specialisation for '0' to terminate the sequence.

which gives us:

1
2
3
4
5
6
7
8

1>c:\projects\fizzbuzz_example\main.cpp(306): error C2039: 'compilation_error_here' : is not a member of 'vector<T0,T1,T2,T3>'
1>          with
1>          [
1>              T0=int_<0>,
1>              T1=int_<1>,
1>              T2=int_<2>,
1>              T3=int_<3>
1>          ]

That looks a lot like what we want, but so far we havn't output Fizz or Buzz; we'll fix that now…

Selecting Fizz

We need to select a different type (Fizz) if the number is divisible by 3. This calculation is a compile time constant for each template instantiation, so we can use it as a parameter to a template; if_c is a fairly simple type selection template that works by specializing for one value of the condition (in this case 'false'):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

template <class T1, class T2, bool c>
struct if_c_impl
{
    typedef T1 type;
};

template <class T1, class T2>
struct if_c_impl<T1, T2, false>
{
    typedef T2 type;
};

template<bool c, class T1, class T2>
struct if_c : if_c_impl<T1,T2,c>
{};

Now we can add the condition for Fizz: 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

struct Fizz{};

template<int i>
struct RunFizzBuzz
{
    typedef int<i> Number;
    typedef if_c<i % 3 == 0, Fizz, Number> condition3;
    typedef typename push_back<typename RunFizzBuzz<i - 1>::type, typename condition3::type>::type type;
};

template<>
struct RunFizzBuzz<0>
{
    typedef vector<int<0> > type;
};

, which produces:

1
2
3
4
5
6
7
8

1>c:\projects\fizzbuzz_example\main.cpp(327): error C2039: 'compilation_error_here' : is not a member of 'vector<T0,T1,T2,T3>'
1>          with
1>          [
1>              T0=int_<0>,
1>              T1=int_<1>,
1>              T2=int_<2>,
1>              T3=Fizz
1>          ]

which is exactly what we want!

Adding the additional conditions for Buzz and FizzBuzz is trivial (as long as we remember to test for FizzBuzz in the right place!):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

struct Fizz{};
struct Buzz{};
struct FizzBuzz{};

template<int i>
struct RunFizzBuzz
{
    typedef int_<i> Number;
    typedef typename if_c<i % 3 == 0, Fizz, Number>::type condition1;
    typedef typename if_c<(i % 5 == 0), Buzz, condition1>::type condition2;
    typedef typename if_c<(i % 3 == 0) && (i % 5 == 0), FizzBuzz, condition2>::type condition3;
    typedef typename push_back<typename RunFizzBuzz<i - 1>::type, condition3>::type type;
};

template<>
struct RunFizzBuzz<0>
{
    typedef vector<int_<0> > type;
};

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

1>c:\projects\fizzbuzz_example\main.cpp(305): error C2039: 'compilation_error_here' : is not a member of 'vector<T0,T1,T2,T3,T4,T5,T6,T7,T8,T9,T10,T11,T12,T13,T14,T15>'
1>          with
1>          [
1>              T0=int_<0>,
1>              T1=int_<1>,
1>              T2=int_<2>,
1>              T3=Fizz,
1>              T4=int_<4>,
1>              T5=Buzz,
1>              T6=Fizz,
1>              T7=int_<7>,
1>              T8=int_<8>,
1>              T9=Fizz,
1>              T10=Buzz,
1>              T11=int_<11>,
1>              T12=Fizz,
1>              T13=int_<13>,
1>              T14=int_<14>,
1>              T15=FizzBuzz
1>          ]

Just for completeness, it works on gcc (MingW) too, although it is a little harder to see…:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

$ gcc --version
gcc.exe (GCC) 4.8.1
Copyright (C) 2013 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ make
g++ -c -Wall main.cpp -o main.o
main.cpp: In function 'int main()':
main.cpp:306:13: error: 'compilation_error_here' in 'RunFizzBuzz<15>::type {aka struct vector<int_<0>, int_<1>, int_<2>,
 Fizz, int_<4>, Buzz, Fizz, int_<7>, int_<8>, Fizz, Buzz, int_<11>, Fizz, int_<13>, int_<14>, FizzBuzz>}' does not name
a type
     typedef RunFizzBuzz<15>::type::compilation_error_here res;
             ^
make: *** [main.o] Error 1

You can find the code used in this article here.

]]> <![CDATA[Using Ruby Installer to install Redcarpet and Github-linguist with Octopress on Windows]]> 2013-09-30T17:32:00+01:00 http://graoil.co.uk/blog/2013/09/30/using-rubyinstaller-to-install-redcarpet-with-octopress-on-windows This article follows three posts describing how to install Ruby 1.9.3 to use Octopress with redcarpet and github-linguist in a native MingW environment; using what I learnt in that process I've figured out how to use the Ruby Installer for Windows to do the same thing - this post describes how to do that. I plan to leave the original posts (starting here) in case they may provide inspiration for someone else with similar ambitions.

I wanted to use redcarpet/github-linguist for markdown in Octopress, however there's a dependency on the charlock_holmes Gem that requires native code libraries (file, ICU), some of which it needs to build. It turns out that we can do this with Ruby Installer

First, install Ruby…

Install and install ruby 1.9.3 (from rubyinstaller.org) Download the devkit (current version here) Unpack dev kit, e.g. into C:\Ruby193\devkit Setup the Ruby installation to work with the devkit: 

1
2

ruby dk.rb init
ruby dk.rb install
All of this follows the regular instructions for installing Ruby and the devkit. <!– more –>

Install Python 2.7

Next, install Python 2.7, if you havn't already - this is required for pygments.rb. Make sure Python is in your path. Check in a Mingw window:

1

python --version

Additional tools for devkit

For some reason, the devkit doesn't come with tar, zip & patch so install these from MingW downloads. I used 7-zip to unpack the wget and xz and then used the binaries of those tools to get the rest; if you want you can just download the missing tools and unpack them with 7-zip, then copy the binaries manually.

If you want to use MingW, first you need to download and unpack xz, wget and dlls for openssl and liblzma; download wget from here , xz from here, openssl dlls from here and liblzma dlls from here. Unpack them all with 7-zip (unpack the tar files as well) and copy the binaries to the bin, etc and lib directories of your devkit (copying with Windows Explorer is fine).

Start the devkit using the msys.bat file located in the devkit root and check that they installed: 

1
2
3
4
5
6

$ wget --version
GNU Wget 1.12 built on msys.
...
$ xz --version
xz (XZ Utils) 5.0.3
liblzma 5.0.3
Now you should be able to use MingW/MSYS to download and install the missing tools; create a temporary directory to unpack the files into and copy the files: 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

cd ~
mkdir tmp
cd tmp
wget http://sourceforge.net/projects/mingw/files/MSYS/Base/tar/tar-1.22-1/tar-1.22-1-msys-1.0.11-bin.tar.lzma
xz -d tar-1.22-1-msys-1.0.11-bin.tar.lzma
tar xvf tar-1.22-1-msys-1.0.11-bin.tar
wget http://sourceforge.net/projects/mingw/files/MSYS/Extension/zip/zip-3.0-1/zip-3.0-1-msys-1.0.14-bin.tar.lzma
xz -d zip-3.0-1-msys-1.0.14-bin.tar.lzma
tar xvf zip-3.0-1-msys-1.0.14-bin.tar
wget http://sourceforge.net/projects/mingw/files/MSYS/Extension/unzip/unzip-6.0-1/unzip-6.0-1-msys-1.0.13-bin.tar.lzma
xz -d unzip-6.0-1-msys-1.0.13-bin.tar.lzma
tar xvf unzip-6.0-1-msys-1.0.13-bin.tar
wget http://sourceforge.net/projects/mingw/files/MSYS/Extension/patch/patch-2.5.9-1/patch-2.5.9-1-msys-1.0.11-bin.tar.lzma
xz -d patch-2.5.9-1-msys-1.0.11-bin.tar.lzma
tar xvf patch-2.5.9-1-msys-1.0.11-bin.tar
wget http://sourceforge.net/projects/mingw/files/MSYS/Base/gzip/gzip-1.3.12-1/gzip-1.3.12-1-msys-1.0.11-bin.tar.lzma
xz -d gzip-1.3.12-1-msys-1.0.11-bin.tar.lzma
tar xvf gzip-1.3.12-1-msys-1.0.11-bin.tar
cp -r -v bin /
cp -r -v share /
cd ..
Now we're ready to install the dependencies for the charlock_holmes Gem…

Build native libraries

We need some additional libraries for the Ruby Gem charlock_holmes. I've built them both with static linking here so that we don't need to worry about where the DLLs might have gone.

Download and build ICU (this post on compiling for qt and this post for compiling for OpenTTD both help): 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

cd ~
mkdir build32
cd ~/build32
wget http://download.icu-project.org/files/icu4c/4.6/icu4c-4_6-src.zip
unzip icu4c-4_6-src.zip
wget http://devs.openttd.org/~terkhen/libicu/libicu_4_6_mingw32.diff
wget http://devs.openttd.org/~terkhen/libicu/libicu_reduce_icudata_size.diff
cd icu
patch -p1 -i ../libicu_4_6_mingw32.diff
patch -p1 -i ../libicu_reduce_icudata_size.diff
cd source
./configure --prefix=/mingw --enable-static --disable-shared --disable-strict --disable-threads
#./configure --prefix=$PWD/../dist --enable-static --disable-shared --disable-strict --disable-threads
make && make install
cd ../..

Download and install libgnurx (required for file-5.08, also a dependency of charlock_holmes): 

1
2
3
4

cd ~/build32
wget http://sourceforge.net/projects/mingw/files/Other/UserContributed/regex/mingw-regex-2.5.1/mingw-libgnurx-2.5.1-src.tar.gz
tar zxvf mingw-libgnurx-2.5.1-src.tar.gz
cd mingw-libgnurx-2.5.1
Apply the following patch for static build: (diff -up Makefile.in Makefile.in.new > ../mingw-libgnurx-static.patch) 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

--- Makefile.in 2007-05-07 20:28:28 +0100
+++ Makefile.in.new     2013-09-23 12:44:15 +0100
@@ -56,7 +56,7 @@ SRCDIST_FILES = ${srcdir}/configure ${sr
 ZIPCMD = @ZIPCMD@
 ZIPEXT = @ZIPEXT@

-all: libgnurx-$(DLLVERSION).dll libgnurx.dll.a libregex.a @GNURX_LIB@
+all: libgnurx.a libgnurx-$(DLLVERSION).dll libgnurx.dll.a libregex.a @GNURX_LIB@

 Makefile: config.status Makefile.in
        ./config.status
@@ -69,6 +69,10 @@ $(OBJECTS): Makefile
 libgnurx-$(DLLVERSION).dll libgnurx.dll.a: $(OBJECTS)
        $(CC) $(CFLAGS) -shared -o libgnurx-$(DLLVERSION).dll $(LDFLAGS) $(OBJECTS)

+libgnurx.a: $(OBJECTS)
+       ar rcu $@ $(OBJECTS)
+       ranlib $@
+
 libregex.a: libgnurx.dll.a
        cp -p libgnurx.dll.a $@

@@ -81,6 +85,11 @@ install-dll:
        mkdir -p ${bindir}
        cp -p $(BINDIST_FILES) ${bindir}

+install-static: libgnurx.a
+       mkdir -p ${includedir} ${libdir}
+       cp -p ${srcdir}/regex.h ${includedir}
+       cp -p ${srcdir}/libgnurx.a ${libdir}
+
 install-dev:
        mkdir -p ${includedir} ${libdir}
        cp -p ${srcdir}/regex.h ${includedir}
Build and install: 
1
2
3

patch -i ../mingw-libgnurx-static.patch
./configure -prefix=/mingw
make && make install-static
Optional: check that file-5.08 builds (note: no make install) without errors: 
1
2
3
4
5
6

cd ~/build32
wget ftp://ftp.astron.com/pub/file/file-5.08.tar.gz
tar zxvf file-5.08.tar.gz
cd file-5.08
./configure --prefix=/mingw  --disable-shared --enable-static --with-pic
make

Install Octopress

Download Octopress (see also Octopress setup instructions). The easiest way to do this is to first install Git for Windows. Start Git Bash, and then (note: this example uses ~/build32 but you can put octopress where you like…): 

1
2

cd /c/Ruby193/devkit/home/[user]/build32/
git clone git://github.com/imathis/octopress.git octopress

At this stage (back in MingW build environment): 

1
2
3
4
5
6
7
8
9
10
11

cd ~/build32/octopress
$ gem list

*** LOCAL GEMS ***

bigdecimal (1.1.0)
io-console (0.3)
json (1.5.5)
minitest (2.5.1)
rake (0.9.2.2)
rdoc (3.9.5)

Install bundler and the default bundle: 

1

gem install bundler
1
2
3
4
5
6
7
8
9
10
11

$ gem list

*** LOCAL GEMS ***

bigdecimal (1.1.0)
bundler (1.3.5)
io-console (0.3)
json (1.5.5)
minitest (2.5.1)
rake (0.9.2.2)
rdoc (3.9.5)
Then install the default bundle: 
1

bundle install
Install the default Octopress theme, create a new test page and generate the site: 
1
2
3

rake install
rake new_post["first post"]
rake generate
You should get the site built in octopress/public. Check that index.html is generated properly (size > 0).

At this point we have the default Octopress installed using rdiscount for markdown generation and we've setup the required libraries for redcarpet. Now we need to install charlock_holmes manually:

Install some updates required for github-linguist

Update the Gemfile to install pygments 0.3.7 - this is required so that we have the updated lexer Augeas (part contents of Gemfile): 

1

gem 'pygments.rb', '~> 0.3.7'
and update the bundle: 
1
2

bundle update pygments.rb
gem install rake-compiler
Now it should look something like this: 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

$ bundle list
Gems included by the bundle:
  * RedCloth (4.2.9)
  * bundler (1.3.5)
  * chunky_png (1.2.5)
  * classifier (1.3.3)
  * compass (0.12.2)
  * directory_watcher (1.4.1)
  * fast-stemmer (1.0.1)
  * fssm (0.2.9)
  * haml (3.1.7)
  * jekyll (0.12.0)
  * kramdown (0.13.8)
  * liquid (2.3.0)
  * maruku (0.6.1)
  * posix-spawn (0.3.6)
  * pygments.rb (0.3.7)
  * rack (1.5.2)
  * rack-protection (1.5.0)
  * rake (0.9.2.2)
  * rb-fsevent (0.9.1)
  * rdiscount (2.0.7.3)
  * rubypants (0.2.0)
  * sass (3.2.9)
  * sass-globbing (1.0.0)
  * sinatra (1.4.2)
  * stringex (1.4.0)
  * syntax (1.0.0)
  * tilt (1.3.7)
  * yajl-ruby (1.1.0)

Create a patch file for pygments.rb: 

1
2
3
4
5
6
7
8
9
10
11

--- popen.rb 2013-09-24 14:41:22 +0100
+++ popen.rb.new  2013-09-24 15:12:42 +0100
@@ -110,7 +110,7 @@
     def lexers
       begin
         lexerfile = File.expand_path('../../../lexers', __FILE_)
-        raw = File.open(lexer_file, "r").read
+        raw = File.open(lexer_file, "rb").read
         Marshal.load(raw)
       rescue Errno::ENOENT
         raise MentosError, "Error loading lexer file. Was it created and vendored?"

Now apply the patch to popen.rb in pygments.rb: 

1
2
3

cd ~/local32/lib/ruby/gems/1.9.1/gems/pygments.rb-0.3.7/lib/pygments
patch -u -i ~/build32/pygments.rb.popen.rb.patch
cd ~/build32/octopress

Get and patch charlock_holmes

1
2
3
4
5
6
7

mkdir ~/build32/tmpgems
cd ~/build32/tmpgems
gem fetch charlock_holmes
gem spec charlock_holmes-0.6.9.4.gem --ruby > charlock_holmes.gemspec
gem unpack charlock_holmes-0.6.9.4.gem
mv -v charlock_holmes.gemspec charlock_holmes-0.6.9.4
cd charlock_holmes-0.6.9.4
Now modify ext/charlock_holmes/extconf.rb: 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

--- extconf.rb   2013-09-24 14:49:14 +0100
+++ extconf.rb.new    2013-09-24 14:51:54 +0100
@@ -9,7 +9,7 @@
   ret
 end
 
-if `which make`.strip.empty?
+if `sh which make`.strip.empty?
   STDERR.puts "\n\n"
   STDERR.puts "***************************************************************************************"
   STDERR.puts "*************** make required (apt-get install make build-essential) =( ***************"
@@ -37,7 +37,8 @@
   end
 end
 
-unless have_library 'icui18n' and have_header 'unicode/ucnv.h'
+#unless have_library 'icui18n' and have_library 'icuuc' and have_library 'regex' and have_header 'unicode/ucnv.h'
+unless have_library 'icudata' and have_library 'icuuc' and have_library 'icui18n' and have_library 'gnurx' and have_header 'unicode/ucnv.h'
   STDERR.puts "\n\n"
   STDERR.puts "***************************************************************************************"
   STDERR.puts "*********** icu required (brew install icu4c or apt-get install libicu-dev) ***********"
@@ -57,7 +58,7 @@
 
   sys("tar zxvf #{src}")
   Dir.chdir(dir) do
-    sys("./configure --prefix=#{CWD}/dst/ --disable-shared --enable-static --with-pic")
+    sys("sh ./configure --prefix=#{CWD}/dst/ --disable-shared --enable-static --with-pic")
     sys("patch -p0 < ../file-soft-check.patch")
     sys("make -C src install")
     sys("make -C magic install")
1
2
3
4
5

cd ext/charlock_holmes
patch -u -i /build32/charlock_holmes_extconf.rb.patch
cd ../..
gem build charlock_holmes.gemspec
gem install charlock_holmes-0.6.9.4.gem --platform=ruby

We should now have charlock_holmes installed: 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

$ gem list

*** LOCAL GEMS ***

bigdecimal (1.1.0)
bundler (1.3.5)
charlock_holmes (0.6.9.4)
chunky_png (1.2.5)
classifier (1.3.3)
compass (0.12.2)
directory_watcher (1.4.1)
fast-stemmer (1.0.1)
fssm (0.2.9)
haml (3.1.7)
io-console (0.3)
jekyll (0.12.0)
json (1.5.5)
kramdown (0.13.8)
liquid (2.3.0)
maruku (0.6.1)
minitest (2.5.1)
posix-spawn (0.3.6)
pygments.rb (0.3.7, 0.3.4)
rack (1.5.2)
rack-protection (1.5.0)
rake (0.9.2.2)
rake-compiler (0.9.1)
rb-fsevent (0.9.1)
rdiscount (2.0.7.3)
rdoc (3.9.5)
RedCloth (4.2.9 x86-mingw32)
rubypants (0.2.0)
sass (3.2.9)
sass-globbing (1.0.0)
sinatra (1.4.2)
stringex (1.4.0)
syntax (1.0.0)
tilt (1.3.7)
yajl-ruby (1.1.0 x86-mingw32)

Add redcarpet to Octopress

1

cd ~/build32/octopress

Now we can modify the Gemfile to include redcarpet (part contents of Gemfile): 

1
2
3
4
5

  gem 'rdiscount', '~> 2.0.7'
  gem "charlock_holmes", "~> 0.6.9.4"
  gem "github-linguist", "~> 2.4"
  gem 'redcarpet', '~> 2.1.1'
  gem 'pygments.rb', '~> 0.3.7'
And update the Gems: 
1
2

cd ~/build32/octopress
bundle install

Test. Save this code into a file in (for example) /tmp/test.rb: 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

require 'rdiscount'
require 'charlockholmes'
require 'escape_utils'
require 'pygments'
require 'yaml'
require 'linguist/classifier'
require 'linguist/samples'
require 'linguist/file_blob'
require 'redcarpet'

puts "Ruby test"

#puts Pygments.highlight(File.read(FILE_), :lexer => 'ruby')

@rdiscount_extensions = "smart"

content = "Some text"
rd = RDiscount.new(content, @rdiscount_extensions)
html = rd.to_html

puts html

detection = CharlockHolmes::EncodingDetector.detect(html)

puts detection

fb = Linguist::FileBlob.new("c:\Ruby193\devkit\dk.rb")
puts fb.language.name

markdown = Redcarpet::Markdown.new(Redcarpet::Render::HTML, :autolink => true, :space_after_headers => true)
puts markdown.render("This is *bongos c++ int main() { return 0 }, indeed.")

And run the test: 

1
2
3
4
5
6
7

$ ruby test.rb
...
Ruby test
<p>Some <em>text</em></p>
{:type=>:text, :encoding=>"ISO-8859-2", :confidence=>40, :language=>"cs"}
Ruby
<p>This is <em>bongos</em> <code>c++ int main() { return 0 }</code>, indeed.</p>

Now modify _config.yml for octopress to generate redcarpet markdown: 

1
2
3

markdown: redcarpet
redcarpet:
  extensions: ["no_intra_emphasis", "fenced_code_blocks", "autolink", "tables", "with_toc_data"]

Modify the 'frist post' created above to include some github markdown, e.g. (note: replace last [backtick]…): 

1

Some text
c++ int main() { return 0; } “[backtick]

Some more text. 

1
2


And regenerate:
rake generate 
1

You should now have syntax highlighted code blocks, like this:
c++ int main() { return 0; } “`

]]> <![CDATA[Installing Ruby 1.9.3 in Windows with Redcarpet and Github Markdown]]> 2013-09-27T17:32:00+01:00 http://graoil.co.uk/blog/2013/09/27/installing-octopress-in-windows-with-redcarpet This is the third of three posts describing how to install Ruby 1.9.3 native MingW environment. It's much simpler to use Ruby Installer to do this but if you want all the details of how to do it natively, read on…

If you're wondering why I did this, I wanted to use redcarpet/github-linguist for markdown in Octopress; there's a dependency on the charlock_holmes Gem that requires native code libraries (file, ICU), some of which it needs to build. I couldn't figure out how to make the Ruby Installer do this and I wanted to understand the process, so I did the whole thing from scratch. The first part is here and the second part is here.

Setup ruby. In the build location (/build32 if following the instructions)…

First, install Python 2.7, if you havn't already - this is required for pygments.rb. Make sure Python is in your path. Check in a Mingw window:

1

python --version

We need some additional libraries for the Ruby Gem charlock_holmes. I've built them both with static linking here so that we don't need to worry about where the DLLs might have gone. <!– more –>

Download and build ICU (this post on compiling for qt and this post for compiling for OpenTTD both help): 

1
2
3
4
5
6
7
8
9
10
11
12
13

cd /build32
wget http://download.icu-project.org/files/icu4c/4.6/icu4c-4_6-src.zip
unzip icu4c-4_6-src.zip
wget http://devs.openttd.org/~terkhen/libicu/libicu_4_6_mingw32.diff
wget http://devs.openttd.org/~terkhen/libicu/libicu_reduce_icudata_size.diff
cd icu
patch -p1 -i ../libicu_4_6_mingw32.diff
patch -p1 -i ../libicu_reduce_icudata_size.diff
cd source
./configure --prefix=/mingw --enable-static --disable-shared --disable-strict --disable-threads
#./configure --prefix=$PWD/../dist --enable-static --disable-shared --disable-strict --disable-threads
make && make install
cd ../..

Download and install libgnurx (required for file-5.08, also a dependency of charlock_holmes): 

1
2
3
4

cd /build32
wget http://sourceforge.net/projects/mingw/files/Other/UserContributed/regex/mingw-regex-2.5.1/mingw-libgnurx-2.5.1-src.tar.gz
tar zxvf mingw-libgnurx-2.5.1-src.tar.gz
cd mingw-libgnurx-2.5.1
Apply the following patch for static build: (diff -up Makefile.in Makefile.in.new > ../mingw-libgnurx-static.patch) 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

--- Makefile.in 2007-05-07 20:28:28 +0100
+++ Makefile.in.new     2013-09-23 12:44:15 +0100
@@ -56,7 +56,7 @@ SRCDIST_FILES = ${srcdir}/configure ${sr
 ZIPCMD = @ZIPCMD@
 ZIPEXT = @ZIPEXT@

-all: libgnurx-$(DLLVERSION).dll libgnurx.dll.a libregex.a @GNURX_LIB@
+all: libgnurx.a libgnurx-$(DLLVERSION).dll libgnurx.dll.a libregex.a @GNURX_LIB@

 Makefile: config.status Makefile.in
        ./config.status
@@ -69,6 +69,10 @@ $(OBJECTS): Makefile
 libgnurx-$(DLLVERSION).dll libgnurx.dll.a: $(OBJECTS)
        $(CC) $(CFLAGS) -shared -o libgnurx-$(DLLVERSION).dll $(LDFLAGS) $(OBJECTS)

+libgnurx.a: $(OBJECTS)
+       ar rcu $@ $(OBJECTS)
+       ranlib $@
+
 libregex.a: libgnurx.dll.a
        cp -p libgnurx.dll.a $@

@@ -81,6 +85,11 @@ install-dll:
        mkdir -p ${bindir}
        cp -p $(BINDIST_FILES) ${bindir}

+install-static: libgnurx.a
+       mkdir -p ${includedir} ${libdir}
+       cp -p ${srcdir}/regex.h ${includedir}
+       cp -p ${srcdir}/libgnurx.a ${libdir}
+
 install-dev:
        mkdir -p ${includedir} ${libdir}
        cp -p ${srcdir}/regex.h ${includedir}
Build and install: 
1
2
3

patch < ../mingw-libgnurx-static.patch
./configure -prefix=/mingw
make && make install-static
Optional: check that file-5.08 builds (note: no make install) without errors: 
1
2
3
4
5
6

cd /build32
wget ftp://ftp.astron.com/pub/file/file-5.08.tar.gz
tar zxvf file-5.08.tar.gz
cd file-5.08
./configure --prefix=/mingw  --disable-shared --enable-static --with-pic
make
Download Octopress (see also Octopress setup instructions). The easiest way to do this is to first install Git for Windows. Start Git Bash, and then (note: this example uses /build32 but you can put octopress where you like…): 
1
2

cd /c/mingw/build32
git clone git://github.com/imathis/octopress.git octopress

At this stage (back in MingW build environment): 

1
2
3
4
5
6
7
8
9
10
11

cd /build32/octopress
$ gem list

*** LOCAL GEMS ***

bigdecimal (1.1.0)
io-console (0.3)
json (1.5.5)
minitest (2.5.1)
rake (0.9.2.2)
rdoc (3.9.5)

Install bundler and the default bundle: 

1

gem install bundler
1
2
3
4
5
6
7
8
9
10
11

$ gem list

*** LOCAL GEMS ***

bigdecimal (1.1.0)
bundler (1.3.5)
io-console (0.3)
json (1.5.5)
minitest (2.5.1)
rake (0.9.2.2)
rdoc (3.9.5)
Then install the default bundle: 
1

bundle install
Update the Gemfile to install pygments 0.3.7 - this is required so that we have the updated lexer Augeas: 
1

gem 'pygments.rb', '~> 0.3.7'
and update the bundle: 
1

bundle update pygments.rb
Now it should look like this: 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

$ bundle list
Gems included by the bundle:
  * RedCloth (4.2.9)
  * bundler (1.3.5)
  * chunky_png (1.2.5)
  * classifier (1.3.3)
  * compass (0.12.2)
  * directory_watcher (1.4.1)
  * fast-stemmer (1.0.1)
  * fssm (0.2.9)
  * haml (3.1.7)
  * jekyll (0.12.0)
  * kramdown (0.13.8)
  * liquid (2.3.0)
  * maruku (0.6.1)
  * posix-spawn (0.3.6)
  * pygments.rb (0.3.7)
  * rack (1.5.2)
  * rack-protection (1.5.0)
  * rake (0.9.2.2)
  * rb-fsevent (0.9.1)
  * rdiscount (2.0.7.3)
  * rubypants (0.2.0)
  * sass (3.2.9)
  * sass-globbing (1.0.0)
  * sinatra (1.4.2)
  * stringex (1.4.0)
  * syntax (1.0.0)
  * tilt (1.3.7)
  * yajl-ruby (1.1.0)

Create a patch file for pygments.rb: 

1
2
3
4
5
6
7
8
9
10
11

--- popen.rb 2013-09-24 14:41:22 +0100
+++ popen.rb.new  2013-09-24 15:12:42 +0100
@@ -110,7 +110,7 @@
     def lexers
       begin
         lexerfile = File.expand_path('../../../lexers', __FILE_)
-        raw = File.open(lexer_file, "r").read
+        raw = File.open(lexer_file, "rb").read
         Marshal.load(raw)
       rescue Errno::ENOENT
         raise MentosError, "Error loading lexer file. Was it created and vendored?"

Now apply the patch to popen.rb in pygments.rb: 

1
2
3

cd /local32/lib/ruby/gems/1.9.1/gems/pygments.rb-0.3.7/lib/pygments
patch -u -i /build32/pygments.rb.popen.rb.patch
cd /build32/octopress
Create a file named 'rake' in /local32/bin. This looks like the central section of rake.bat: 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

#!C:/mingw/local32/bin/ruby

#--
# Copyright (c) 2003, 2004, 2005, 2006, 2007  Jim Weirich
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to
# deal in the Software without restriction, including without limitation the
# rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
# sell copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
# IN THE SOFTWARE.
#++

begin
  require 'rubygems'
rescue LoadError
end

require 'rake'

Rake.application.run
END
Install the default Octopress theme, create a new test page and generate the site: 
1
2
3

rake install
rake new_post["first post"]
rake generate
You should get the site built in octopress/public. Check that index.html is generated properly (size > 0).

At this point we have the default Octopress installed using rdiscount for markdown generation and we've setup the required libraries for redcarpet. Now we need to install charlock_holmes manually:

1
2
3
4
5
6
7
8

gem install rake-compiler
mkdir /build32/tmp
cd /build32/tmp
gem fetch charlock_holmes
gem spec charlock_holmes-0.6.9.4.gem --ruby > charlock_holmes.gemspec
gem unpack charlock_holmes-0.6.9.4.gem
mv -v charlock_holmes.gemspec charlock_holmes-0.6.9.4
cd charlock_holmes-0.6.9.4
Now modify ext/charlock_holmes/extconf.rb: 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

--- extconf.rb   2013-09-24 14:49:14 +0100
+++ extconf.rb.new    2013-09-24 14:51:54 +0100
@@ -9,7 +9,7 @@
   ret
 end
 
-if `which make`.strip.empty?
+if `sh which make`.strip.empty?
   STDERR.puts "\n\n"
   STDERR.puts "***************************************************************************************"
   STDERR.puts "*************** make required (apt-get install make build-essential) =( ***************"
@@ -37,7 +37,8 @@
   end
 end
 
-unless have_library 'icui18n' and have_header 'unicode/ucnv.h'
+#unless have_library 'icui18n' and have_library 'icuuc' and have_library 'regex' and have_header 'unicode/ucnv.h'
+unless have_library 'icudata' and have_library 'icuuc' and have_library 'icui18n' and have_library 'gnurx' and have_header 'unicode/ucnv.h'
   STDERR.puts "\n\n"
   STDERR.puts "***************************************************************************************"
   STDERR.puts "*********** icu required (brew install icu4c or apt-get install libicu-dev) ***********"
@@ -57,7 +58,7 @@
 
   sys("tar zxvf #{src}")
   Dir.chdir(dir) do
-    sys("./configure --prefix=#{CWD}/dst/ --disable-shared --enable-static --with-pic")
+    sys("sh ./configure --prefix=#{CWD}/dst/ --disable-shared --enable-static --with-pic")
     sys("patch -p0 < ../file-soft-check.patch")
     sys("make -C src install")
     sys("make -C magic install")
1
2
3
4
5

cd ext/charlock_holmes
patch -u -i /build32/charlock_holmes_extconf.rb.patch
cd ../..
gem build charlock_holmes.gemspec
gem install charlock_holmes-0.6.9.4.gem --platform=ruby

We should now have charlock_holmes installed: 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

$ gem list

*** LOCAL GEMS ***

bigdecimal (1.1.0)
bundler (1.3.5)
charlock_holmes (0.6.9.4)
chunky_png (1.2.5)
classifier (1.3.3)
compass (0.12.2)
directory_watcher (1.4.1)
fast-stemmer (1.0.1)
fssm (0.2.9)
haml (3.1.7)
io-console (0.3)
jekyll (0.12.0)
json (1.5.5)
kramdown (0.13.8)
liquid (2.3.0)
maruku (0.6.1)
minitest (2.5.1)
posix-spawn (0.3.6)
pygments.rb (0.3.7, 0.3.4)
rack (1.5.2)
rack-protection (1.5.0)
rake (0.9.2.2)
rake-compiler (0.9.1)
rb-fsevent (0.9.1)
rdiscount (2.0.7.3)
rdoc (3.9.5)
RedCloth (4.2.9 x86-mingw32)
rubypants (0.2.0)
sass (3.2.9)
sass-globbing (1.0.0)
sinatra (1.4.2)
stringex (1.4.0)
syntax (1.0.0)
tilt (1.3.7)
yajl-ruby (1.1.0 x86-mingw32)
cd /build32/octopress

Now we can modify the Gemfile to include redcarpet: 

1
2
3
4
5

  gem 'rdiscount', '~> 2.0.7'
  gem "charlock_holmes", "~> 0.6.9.4"
  gem "github-linguist", "~> 2.4"
  gem 'redcarpet', '~> 2.1.1'
  gem 'pygments.rb', '~> 0.3.7'
And update the Gems: 
1
2

cd /build32/octopress
bundle install

Test. Save this code into a file in (for example) /tmp/test.rb: 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

require 'rdiscount'
require 'charlockholmes'
require 'escape_utils'
require 'pygments'
require 'yaml'
require 'linguist/classifier'
require 'linguist/samples'
require 'linguist/file_blob'
require 'redcarpet'

puts "Ruby test"

#puts Pygments.highlight(File.read(FILE_), :lexer => 'ruby')

@rdiscount_extensions = "smart"

content = "Some text"
rd = RDiscount.new(content, @rdiscount_extensions)
html = rd.to_html

puts html

detection = CharlockHolmes::EncodingDetector.detect(html)

puts detection

fb = Linguist::FileBlob.new("c:\Ruby193\devkit\dk.rb")
puts fb.language.name

markdown = Redcarpet::Markdown.new(Redcarpet::Render::HTML, :autolink => true, :space_after_headers => true)
puts markdown.render("This is *bongos c++ int main() { return 0 }, indeed.")

And run the test: 

1
2
3
4
5
6
7

$ ruby test.rb
...
Ruby test
<p>Some <em>text</em></p>
{:type=>:text, :encoding=>"ISO-8859-2", :confidence=>40, :language=>"cs"}
Ruby
<p>This is <em>bongos</em> <code>c++ int main() { return 0 }</code>, indeed.</p>

Now modify _config.yml for octopress to generate redcarpet markdown: 

1
2
3

markdown: redcarpet
redcarpet:
  extensions: ["no_intra_emphasis", "fenced_code_blocks", "autolink", "tables", "with_toc_data"]

Modify the post created above to include some github markdown, e.g. (note: replace last [backtick]…): 

1

Some text
c++ int main() { return 0; } “[backtick]

Some more text. 

1
2


And regenerate:
rake generate “` You should now have syntax highlighted code blocks!

]]> <![CDATA[Installing Ruby 1.9.3 in Windows with MingW]]> 2013-09-26T17:32:00+01:00 http://graoil.co.uk/blog/2013/09/26/installing-ruby-in-windows This is the second of three posts describing how to install Ruby 1.9.3 native MingW environment. It's much simpler to use Ruby Installer to do this but if you want all the details of how to do it natively, read on…

If you 're wondering why I did this, I wanted to use redcarpet/github-linguist for markdown in Octopress; there's a dependency on the charlock_holmes Gem that requires native code libraries (file, ICU), some of which it needs to build. I couldn't figure out how to make the Ruby Installer do this and I wanted to understand the process, so I did the whole thing from scratch. The first part is here and the final part is here.

Setup Msys base system.

Download and build libyaml. Yaml requires a built DLL and needs it to be available so we build that manually (instructions for hacking a ruby gem very useful knowledge for this). Not sure if the first part to build and install the static libraries is required. We're hard coding the target location, but just need to get it to work…: 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

cd /build32
wget http://pyyaml.org/download/libyaml/yaml-0.1.4.tar.gz
tar zxvf yaml-0.1.4.tar.gz
cd yaml-0.1.4
./configure --prefix=/mingw CFLAGS="-DYAML_DECLARE_STATIC" 
make && make install
cd src
gcc -c -I../include -I../ -DHAVE_CONFIG_H -DYAML_DECLARE_EXPORT api.c
gcc -c -I../include -I../ -DHAVE_CONFIG_H -DYAML_DECLARE_EXPORT dumper.c
gcc -c -I../include -I../ -DHAVE_CONFIG_H -DYAML_DECLARE_EXPORT emitter.c
gcc -c -I../include -I../ -DHAVE_CONFIG_H -DYAML_DECLARE_EXPORT loader.c
gcc -c -I../include -I../ -DHAVE_CONFIG_H -DYAML_DECLARE_EXPORT parser.c
gcc -c -I../include -I../ -DHAVE_CONFIG_H -DYAML_DECLARE_EXPORT reader.c
gcc -c -I../include -I../ -DHAVE_CONFIG_H -DYAML_DECLARE_EXPORT scanner.c
gcc -c -I../include -I../ -DHAVE_CONFIG_H -DYAML_DECLARE_EXPORT writer.c
gcc -shared -o yaml.dll api.o dumper.o emitter.o loader.o parser.o reader.o scanner.o writer.o -Wl,--out-implib,libyaml.dll.a -Wl,--output-def,libyaml.def
cp yaml.dll /mingw/bin
cp libyaml.def libyaml.dll.a /mingw/lib
cd ..
<!– more –> Download and build libffi: 
1
2
3
4
5
6
7
8

cd /build32
wget ftp://sourceware.org/pub/libffi/libffi-3.0.13.tar.gz
wget http://www.linuxfromscratch.org/patches/blfs/svn/libffi-3.0.13-includedir-1.patch
tar zxvf libffi-3.0.13.tar.gz
cd libffi-3.0.13
patch -Np1 < ../libffi-3.0.13-includedir-1.patch
./configure --prefix=/mingw
make && make install
Download and build gdbm 
1
2
3
4
5
6
7
8

cd /build32
wget ftp://ftp.gnu.org/gnu/gdbm/gdbm-1.10.tar.gz
tar zxvf gdbm-1.10.tar.gz
wget https://raw.github.com/luislavena/knapsack-recipes/master/gdbm/1.10/0001-Mingw-port-of-gdbm-1.10.patch --no-check-certificate
cd gdbm-1.10
patch -p1 < ../0001-Mingw-port-of-gdbm-1.10.patch
./configure --prefix=/mingw --enable-shared
make && make install
Download and build termcap: 
1
2
3
4
5
6

cd /build32
wget http://ftp.gnu.org/gnu/termcap/termcap-1.3.1.tar.gz
tar zxvf termcap-1.3.1.tar.gz
cd termcap-1.3.1
./configure --prefix=/mingw --enable-static
make && make install
Download and build readline: 
1
2
3
4

cd /build32
wget ftp://ftp.cwru.edu/pub/bash/readline-6.2.tar.gz
tar zxvf readline-6.2.tar.gz
cd readline-6.2
Patch rldefs.h and util.c to make params const char. Patch _rl_strnicmp in util.c so that do{}while has trailing ';': 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

diff -rupN readline-6.2.orig//rldefs.h readline-6.2//rldefs.h
--- readline-6.2.orig//rldefs.h   2009-01-04 19:32:33 +0000
+++ readline-6.2//rldefs.h    2013-09-19 09:38:32 +0100
@@ -79,8 +79,8 @@ extern char *strchr (), *strrchr ();
 #define _rl_stricmp strcasecmp
 #define _rl_strnicmp strncasecmp
 #else
-extern int _rl_stricmp PARAMS((char *, char *));
-extern int _rl_strnicmp PARAMS((char *, char *, int));
+extern int _rl_stricmp PARAMS((const char *, const char *));
+extern int _rl_strnicmp PARAMS((const char *, const char *, int));
 #endif
 
 #if defined (HAVE_STRPBRK) && !defined (HAVE_MULTIBYTE)
diff -rupN readline-6.2.orig//util.c readline-6.2//util.c
--- readline-6.2.orig//util.c 2010-05-30 23:36:02 +0100
+++ readline-6.2//util.c  2013-09-19 09:38:57 +0100
@@ -369,10 +369,10 @@ _rl_strpbrk (string1, string2)
    doesn't matter (strncasecmp). */
 int
 _rl_strnicmp (string1, string2, count)
-     char *string1, *string2;
+     const char *string1, *string2;
      int count;
 {
-  register char *s1, *s2;
+  register const char *s1, *s2;
   int d;
 
   if (count <= 0 || (string1 == string2))
@@ -389,7 +389,7 @@ _rl_strnicmp (string1, string2, count)
         break;
       s2++;
     }
-  while (--count != 0)
+  while (--count != 0);
 
   return (0);
 }
@@ -397,9 +397,9 @@ _rl_strnicmp (string1, string2, count)
 / strcmp (), but caseless (strcasecmp). /
 int
 _rl_stricmp (string1, string2)
-     char *string1, *string2;
+     const char *string1, *string2;
 {
-  register char *s1, *s2;
+  register const char *s1, *s2;
   int d;
 
   s1 = string1;
Build: 
1
2
3

patch -p1 < ../readline-6.2.patch
./configure --prefix=/mingw --enable-static
make && make install
Download and install openssl: 
1
2
3
4
5
6
7
8

cd /build32
wget http://www.openssl.org/source/openssl-1.0.1e.tar.gz
tar zxvf openssl-1.0.1e.tar.gz
cd openssl-1.0.1e
perl Configure mingw no-shared no-asm --prefix=/mingw
make depend
make
make install
Download and install ruby (note: dbm, pty and syslog will fail to configure, this is ok on Windows…no others should fail). For some reason we need the drive/path to the install location or it doesn't install any files…: 
1
2
3
4
5
6
7

cd /build32
wget http://cache.ruby-lang.org/pub/ruby/1.9/ruby-1.9.3-p448.tar.gz
tar zxvf ruby-1.9.3-p448.tar.gz
cd ruby-1.9.3-p448
./configure --prefix=C:/mingw/local32 --enable-shared --disable-install-doc
make
make install
Test…should output the version, e.g.: 
1
2

$ ruby -v
ruby 1.9.3p448 (2013-06-27 revision 41675) [i386-mingw32]
Download and install rubygems: 
1
2
3
4
5

cd /build32
wget http://production.cf.rubygems.org/rubygems/rubygems-2.0.7.tgz
tar zxvf rubygems-2.0.7.tgz
cd rubygems-2.0.7
ruby setup.rb
Test (final part of this useful post): 
1
2
3
4
5

$ gem list omniauth$ -r --all

** REMOTE GEMS ***

omniauth (1.1.4, 1.1.3, 1.1.2, 1.1.1, 1.1.0, 1.0.3, 1.0.2, 1.0.1, 1.0.0, 0.3.2, 0.3.0, 0.2.6, 0.2.5, 0.2.4, 0.2.3, 0.2.2, 0.2.1, 0.2.0, 0.1.6, 0.1.5, 0.1.4, 0.1.3, 0.1.2, 0.1.1, 0.1.0, 0.0.5, 0.0.4, 0.0.3, 0.0.1)
It works!

]]> <![CDATA[Setting up Mingw ready for Ruby 1.9.3 install]]> 2013-09-24T21:31:00+01:00 http://graoil.co.uk/blog/2013/09/24/setting-up-mingw This is the first of three posts describing how to install Ruby 1.9.3 native MingW environment. It's much simpler to use Ruby Installer to do this but if you want all the details of how to do it natively, read on…

If you 're wondering why I did this, I wanted to use redcarpet/github-linguist for markdown in Octopress; there's a dependency on the charlock_holmes Gem that requires native code libraries (file, ICU), some of which it needs to build. I couldn't figure out how to make the Ruby Installer do this and I wanted to understand the process, so I did the whole thing from scratch. The second part is here and the final part is here.

This is a prerequisite for installing Ruby 1.9.3 in a native Mingw environment on Windows.

Download and run mingw installer: mingw-get-setup.exe Select 'mingw-developer-toolkit' and apply changes. This installs a default msys and default packages. Start msys.bat (usually in C:\MinGW\msys\1.0).

Edit /etc/fstab to add: 

1
2
3

c:/mingw        /mingw
c:/mingw/local32        /local32
c:/mingw/build32        /build32
<!– more –> Create a directory for app install and a build directory (similar to instructions for installing a base system) 
1
2
3
4
5
6

mkdir /usr/local
mkdir /usr/local/bin
mkdir /c/mingw/opt
mkdir /c/mingw/build32 /c/mingw/local32
mkdir /c/mingw/opt/bin /local32/{bin,etc,include,lib,share}
mkdir /local32/lib/pkgconfig
Create a profile: 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

cat > /local32/etc/profile.local << "EOF"
#
# /local32/etc/profile.local
#

alias dir='ls -la --color=auto'
alias ls='ls --color=auto'

PKG_CONFIG_PATH="/local32/lib/pkgconfig"
CPPFLAGS="-I/local32/include"
CFLAGS="-I/local32/include"
CXXFLAGS="-I/local32/include"
LDFLAGS="-L/local32/lib"
export PKG_CONFIG_PATH CPPFLAGS CFLAGS CXXFLAGS LDFLAGS

PATH="/local32/bin:$PATH"

# package build directory
LOCALBUILDDIR=/build32
# package installation prefix
LOCALDESTDIR=/local32
export LOCALBUILDDIR LOCALDESTDIR

EOF
and a user profile to load it: 
1
2
3
4
5
6

cat >> /etc/profile << "EOF"
if [ -f /local32/etc/profile.local ]; then
        source /local32/etc/profile.local
fi

EOF
Remove unwanted automake's (following these Mingw install instructions): 
1

mingw-get remove automake1.4 automake1.5 automake1.6 automake1.7 automake1.8
'exit' from all msys shells and restart. You should see the following mounts: 
1
2
3
4
5
6

$ mount
...
c:\mingw\build32 on /build32 type user (binmode)
c:\mingw\local32 on /local32 type user (binmode)
c:\mingw on /mingw type user (binmode)
...
Add some additional packages: 
1
2
3

mingw-get install base bzip2 expat gcc-g++ gettext gmp libarchive libpdcurses libpopt libunistring lua mingw-utils mpc mpfr pdcurses pkginfo pthreads-w32 zlib

mingw-get install msys-bash msys-bison msys-core-dev msys-crypt msys-cygutils msys-inetutils msys-unzip msys-wget msys-zip
(optional) Run the mingw installer and mark mingw-tcl and mingw-tk for installation and apply changes. Check they are there: 
1
2

cd /mingw
find . -name "tk.h"

]]> <![CDATA[Time for a blog]]> 2013-09-24T07:45:00+01:00 http://graoil.co.uk/blog/2013/09/24/time-for-a-blog For some time now I've been planning to write a blog; I'd seen Matt Gemmell's "Blogging with Octopress" and his new site design. As a geeky software developer, that looked like just what I wanted. The only problem was getting it installed and setup just how I wanted it; I don't yet have a Mac and I'd tried various Linux flavours without much success.

So I decided to try running it on Windows, and that led to a whole host of learning experiences involving Ruby, Windows and Native Gems that I'll blog about later…

Like Matt, I decided to turn off comments. You can contact me on twitter @ColonelSammy or by email (mnoyes @ this domain).

This should come out right: 

1
2
3
4
5
6
7

#include <iostream>

int main()
{
  std::cout << "Hellow world" << std::endl;
  return 0;
}

Matthews DOE projects

Now to go and write something more interesting…

]]>