Martin's Tex-Blog

Posts on programming and generally technical topics

Linkage of templates

leave a comment »


I would like to share observation of another non standard behavior found in Visual Studio compiler. According to C++ Templates Complete Guide: “class templates cannot share a name with a different kind of entity”, below code:

int X;
template
class X {    
};

gives following error with g++ 4.9:


main.cpp:8:7: error: 'template struct X' redeclared as different kind of symbol
class X {
^
main.cpp:6:5: note: previous declaration 'int X'
int X;

but with VS2013 compiles just fine.

Advertisements

Written by Marcin

December 28, 2014 at 2:24 am

Posted in Uncategorized

Be carefull with subnormal floats

leave a comment »


Subnormal floating point numbers are values that are very small (.0f to ~.1f), operations on them are specially treated by CPUs. Below is a timing of simple sumation of floating point values, you can see that once subnormal values are used, performance instantly drops 10 or more times.

http://coliru.stacked-crooked.com/a/6ffe763ab4b5ada0

// Example code for testing subnormal float efficency
float f = 2.0f;
std::chrono::steady_clock::duration prevdiff;
for (int k = 0; k < 10000; ++k) {
  float sum = .0f;		
  auto start = std::chrono::steady_clock::now();

  for (int n = 0; n < 1e7; ++n) {
    sum += f * 0.1f;  // interesting, but simple summation show no performance degradation
  }

  auto end = std::chrono::steady_clock::now();
  auto diff = end - start;
  std::cout << k << ", f=" << f << ", isnormal=" << std::isnormal(f)
    << ", elapsed=" << std::chrono::duration_cast<std::chrono::milliseconds>(diff).count() << "ms" <<
    ", ratio=" << (prevdiff.count() == 0 ? 0 : (diff.count() / prevdiff.count())) << std::endl;
    prevdiff = diff;

   if (f < std::numeric_limits<float>::min())
     break;

    f /= 2.0f;
}

here is the interesting part from the output:

123, f=1.88079e-37, isnormal=1, elapsed=69ms, ratio=1
124, f=9.40395e-38, isnormal=1, elapsed=799ms, ratio=11 <-------- here performance drop
125, f=4.70198e-38, isnormal=1, elapsed=800ms, ratio=1
126, f=2.35099e-38, isnormal=1, elapsed=800ms, ratio=0
127, f=1.17549e-38, isnormal=1, elapsed=796ms, ratio=0
128, f=5.87747e-39, isnormal=0, elapsed=985ms, ratio=1

whats interesting is that this drop happens before std::isnormal returns true. You can use _MM_SET_FLUSH_ZERO_MODE(_MM_FLUSH_ZERO_ON); to make CPU not do computations using subnormal values.

This post is based on following SO : http://stackoverflow.com/questions/9314534/why-does-changing-0-1f-to-0-slow-down-performance-by-10x.

Written by Marcin

December 21, 2014 at 4:40 pm

Posted in Uncategorized

Dependent base class name lookup (c++ template complete guide)

leave a comment »


If you were reading c++ template complete guide : “5.2 using this->” you probably tried to test following example:

template<typename T>
class Base {
public:
  void exit() {}
};

template<typename T>
class Derived : public Base < T > {
public:
	void callMe() {
	  exit();
	}
};

if you done it under Visual Studio, you have probably realized that it does compile with no errors or warnings, so what is going on here. Your next step should be to test it with gcc or clang compiler, you would see errors now.

The problem is that VS does not implement two phase name lookup. I have found that in VS2013 diabling language extensions with /Za actually causes above code to fail to compile.

some usefull links below
http://blog.llvm.org/2009/12/dreaded-two-phase-name-lookup.html
http://stackoverflow.com/questions/11405/gcc-problem-using-a-member-of-a-base-class-that-depends-on-a-template-argument
http://clang.llvm.org/compatibility.html#dep_lookup_bases

Written by Marcin

December 16, 2014 at 11:24 am

Posted in C++

shared_ptr and aliasing constructor

leave a comment »


shared_ptr seems to be straightforward class, but if you look closer you will find aliasing constructor that make you think – what the hell is that?

template<class Y>
shared_ptr( const shared_ptr<Y>& r, T *ptr );

The idea of this constructor is to keep reference count of owned object – r, while using pointer of some other stored object (mostly field in r object). This means that shared_ptr is allowed to own one object and store some other raw pointer. Only owned object is reference counted, while stored pointer is the one that gets returned during dereferencing. In normal situation, when aliasing constructor is not used, raw pointer is the one for owned object.

Below is example of aliasing constructor usage, where stored pointer is contained inside owned one:

#include <iostream>
#include <memory>

struct Data {
	int d;
};

int main(int argc, char* argv[])
{
	std::shared_ptr<Data> pd(new Data());
	std::shared_ptr<int> pd_d(pd, &pd->d); //pd is owned object, while &pd->d is stored (retuned on dereference)

	*pd_d = 10; 

	std::cout << pd.use_count() << std::endl; // outputs 2
	pd_d.reset();
	std::cout << pd.use_count() << std::endl; // outputs 1
	pd.reset();
	std::cout << pd.use_count() << std::endl; // outputs 0

	// pd->d is now a dangling pointer, dont use it!

	return 0;
}

Other use is for making shared_ptr of derived type:

        struct Base {
	  virtual ~Base(){}
        };
        struct Derived : public Base {
	  void foo() {}
        };

	std::shared_ptr<Base> pd_base(new Derived());
	std::shared_ptr<Derived> pd_derived(pd_base, dynamic_cast<Derived*>(pd_base.get()));
	pd_derived->foo();

Other use case for aliasing constructor is when you are using interface that requires shared_ptr but you dont want it to keep ownership of your object. In such case you can make shared_ptr to work like normal bare pointer:

    static Data globalData;
    std::shared_ptr pd2(std::shared_ptr(), &globalData.d);
    std::cout << pd2.use_count() << std::endl; // outputs 0, but *pd is valid
    *pd2 = 2;

Aliasing constructor is not forcing you to keep pointer as a stored object, it can be any type of size_t. When you realize this you can use this constructor also for keeping timestamp in some caching mechanism. For further info look here:

http://www.codesynthesis.com/~boris/blog/2012/04/25/shared-ptr-aliasing-constructor/

Other uses are: refering to container element (owned object is container, stored pointer is element in container), referring to a shared library symbol.

What you should not do:

	std::shared_ptr pd(new Data());
	std::shared_ptr pd_d(pd, new int()); // here new int() is an error, this pointer will never be freed.

Written by Marcin

November 15, 2014 at 5:19 pm

Posted in C++

gitstats filter files

leave a comment »


Currently gitstats does not allow to filter files by extensions, and the only way is as author suggests:

https://github.com/dcramer/gitstats/blob/master/doc/gitstats.pod
Q: I have files in my git repository that I would like to exclude from the statistics, how do I do that?
A: At the moment the only way is to use git-filter-branch(1) to create a temporary repository and generate the statistics from that.

below is a short script on how to prepeare your repository for such filtered gitstats statistics calcualtion:

i assume you have copied your repository to some temporary folder, actually all below changes can be reverted but to be safe do it in temporary dir.

# clear file list to be removed from statistics calculation
$ rm ~/temp_gitstats_filter.txt

# get latest changes
$ git pull

# create a list of files excluding the ones ending with java and xml, those files will not be processed by gitstats
$ for i in `git ls-files | grep -Ev ‘^.*.(java|xml)$’`; do echo “$i” >> ~/temp_gitstats_filter.txt; done

# remove all the files from previously created list in current repository
$ git filter-branch -f –tree-filter ‘cat ~/temp_gitstats_filter.txt | xargs rm -f’ HEAD

# now execute your gitstats command (should be fixed for your needs)
$ gitstats .git /var/www/html/git_stats

# reset all the changes filter-branch have done
$ git reset –hard refs/original/refs/heads/master

some usefull links:
http://ktln2.org/blog/post/remove-files-from-history-with-filter-branch/
http://git.661346.n2.nabble.com/Remove-all-files-except-a-few-files-using-filter-branch-td7567155.html
http://stackoverflow.com/questions/4228100/git-i-want-to-unstage-all-files-matching-a-certain-pattern
http://unix.stackexchange.com/questions/37313/how-do-i-grep-for-multiple-patterns

Written by Marcin

October 13, 2014 at 9:41 am

Posted in git

rvalue, lvalue, prvalue, xvalue, glvalue – what are those?

leave a comment »


Previous standard C++03 divided expressions into lvalues and rvalues, new standard C++11 adds three more categories: xvalue, glvalue and prvalue. What are those and why we need them all?

Lets start with lvalues and rvalues.

The common definition for lvalue is a non-const object that may be put on the left hand side of assignment. It can be an expression or function.

rvalue on the other hand is an expression that cannot be placed on the left hand side of assignment, it provides only value and cannot be assigned to.

For example:

int lvalue;
lvalue = 1;         // 1*
(lvalue + 10) = 1;  // 2* - error

char arr[15];
*(arr + 3) = 'a' ; // 3*

int& lvalueRetFunc(int& v) { return v; }
int rvalueRetFunc(int v) { return v; }
lvalueRetFunc(lvalue) = 10; // 4*
rvalueRetFunc(lvalue) = 10; // 5* - error

1* – here we have proper assignment to lvalue
2* – here we have an expression (lvalue + 10) that denotes rvalue which cannot be assigned, in VS C++ 2010 this code will produce following error “error C2106: ‘=’ : left operand must be l-value”.
3* – example of expression that yields lvalue
4* – here we have a function returning lvalue which can be assigned
5* – here function returns rvalue which cannot be assigned and results in the same error like in 2*

From those examples it is easy to see that lvalue can be used where rvalue is required, but not the other way around, rvalue cannot be used in place of lvalue with the exception when c++11 new construct – rvalue reference is used.

Fundamental property of lvalue is that it has a location. This location can be used in assignment expressions, you can use lvalue to take its address.

lvalues are also said to be persistent, rvalues on the other hand are temporary, or ephemeral and are available only until end of full expression, but if rvalue it assigned to const reference or pointer then its lifetime will be prolonged.

lvalues not always can be modified, for example when they are const variables or array types.

Now lets introduce xvalue, glvalue and prvalue. With C++11 comes new language feature called rvalue references. It allows to steal internals of rvalues and use them in constructing new objects that can be later on used as lvalues. This was actually possible in C++03 but was largely complicated. Those three new expression categories were introduced to properly describe this new language feature.

Xvalue (eXpiring value) is the most important new concept, it is tightly connected to new move semantics. To shortly describe move semantics, lets see an example:

In C++03 you would initialize class object with string as follows:

class MyData {
std::string s;
public:
  MyData(std::string ss)
   :  s(ss) // (2*) here creates another allocation and copy
  {}
};

MyData md("mydata"); // (1*) creates one temporary which requires at least one allocation

The problem is that it introduces temporary rvalues that allocate memory and also do some other operations. With C++11 comes possibility to actually “steal” inner contents from such temporary in (2*) and eliminate not needed allocations/operations:

class MyData {
std::string s;
public:
  MyData(std::string ss) : s(std::move(ss)) {} // here no allocations are created, inner guts of temporary ss are used to initialize s variable
};

MyData md("mydata"); // here creates temporary string object

Xvalue in the second example is the std::move(ss) experssion, it returns rvalue reference which is an indication to std::string class that we want to steal inner contents from this temporary and use it in s variable. What std::move really do is to turn named variable into unnamed reference, this is an indication to compiler that we really know what we are doing, and we want s variable to be constructed from ss using move semantics. So the new name for such expression was needed, since move semantics in this example is explicit – without std::move we are back in C++03 world.

After that ss in MyData() constructor is actually empty, standard guarantees that such objects are in a “valid but unspecified state”, that means you can assign new values to it, but current value is not defined.

So – since C++11 rvalue got new semantic which is being an rvalue reference – this means that besides its old semantic it can also behave as xvalue described above.

Now to those two last names:
– when you see prvalue (“pure” rvalues) – this indicated plain old rvalues from C++03. Those are not xvalues.
glvalue (“generalized” lvalue) is just a grouping for lvalues and xvalues.

[refs]
http://www.cprogramming.com/c++11/rvalue-references-and-move-semantics-in-c++11.html
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2010/n3055.pdf

Written by Marcin

October 8, 2012 at 2:09 am

Posted in C++

RandomAccessFile does not lock your file by default

leave a comment »


If you use this class for writing then you probably want to ensure that you are the only one who is allowed to modify given file. I recently was fixing code that used RandomAccessFile for writing huge files downloaded from internet, but it didnt lock those file during writing. Whats wrong with that? Actually from the user perspective nothing, those file wont be accessed any way by any other software, but what we are missing here is the fact that only one thread should be allowed to read/write those files is an invariant which should be checked in code to trap any errors that could show up. And if this invariant were checked in the code I was fixing, developer would find his bug much earlier.

In todays heavily multithreaded environments it is easy to introduce errors, thats why invariants should be checked whether possible and exceptions should be thrown always when they are not correctly met.

Below is a small method for checking if file is locked or not.

public static boolean isFileLocked(File f) {
	RandomAccessFile file = null;
	FileChannel fileChannel = null;
	FileLock fileLock = null;

	try {
		file = new RandomAccessFile(f, "r");
		fileChannel = file.getChannel();
		if (fileChannel == null)
			return false;
		fileLock = fileChannel.lock();
	} catch (FileNotFoundException fnfe) {
		return false;
	} catch (Exception e) {
		// ClosedChannelException
		// NonWritableChannelException
		// OverlappingFileLockException
		// FileLockInterruptionException
		// AsynchronousCloseException
		// IOException
		return true;
	} finally {
		try {
			if (fileLock != null)
				fileLock.release();
			if (fileChannel != null)
				fileChannel.close();
			if (file != null)
				file.close();
		} catch (IOException ioe) {
			return false;
		}
	}
	return false;
}

Written by Marcin

October 4, 2012 at 3:31 pm

Posted in Android, Uncategorized