Martin's Tex-Blog

Posts on programming and generally technical topics

Archive for the ‘Uncategorized’ Category

Docker: how to curl from one container into another

leave a comment »


I have a docker compose file that is similar to the following one:

version: "3"
services:
    web:
        image: nginx
        ports:
            - "80:80"
        volumes:
            - ./volumes/www/app1.example.com/html:/var/www/app1.example.com/html:z
            - ./volumes/www/app2.example.com/html:/var/www/app2.example.com/html:z
            - ./volumes/www/app3.example.com/html:/var/www/app3.example.com/html:z
        restart: always
    cron:
      build:
        context: .
        dockerfile: cron-dockerfile
      links:
        - php
        - web
    php:
      # unimportant
    db:
      # unimportant

as you see I have a web container with nginx serving webpages from three virtual hosts. For each I have a domain name specific to application name. I also have a cron container which is supposed to do some maintance jobs from time to time. It does it by executing some php script on web server. Before migrating to docker I used basicly:

 curl http://app1.example.com/some_maintance.php 

But with docker I always get host unreachable error.

After some investigation I found that I am able to ping app1.example.com from cron container but curl or wget does not reach it, also web which is supposed to be a link to nginx container from cron container has different ip than actual app1.example.com (for which I have registrated domain with physical ip address). I have access to any other domain names – http://www.google.com etc. is accessible by curl.

So the first solution I tried was to update \etc\hosts on cron container, I added 1.2.3.4 app1.example.com entry, where 1.2.3.4 is IP address of web container. And it worked. The problem is that from what I know modifying \etc\hosts is not encouraged by docker community. Instead you should use extra_hosts in docker compose yaml file, but you can specify in extra_hosts only explicit IP addresses, you cannot use container name (like web in my configuration). From what I know, if you need such feature then it is a sign you should start using custom networks. I tried to use custom networks but unfortunately with no luck.

Finally what worked was to make curl use IP address of my web container (where I can put web), and then pass domain name as -H parameter. So the command looks as follows:

curl -H'Host: app1.example.com' web/some_maintance.php
Advertisements

Written by Marcin

September 19, 2017 at 3:48 pm

Posted in Uncategorized

Tagged with

Platform dependend overload resolution ambiguity

leave a comment »


Here I will show case when types used internally by compiler might cause ambiguities in overload resolution. Below is example code:

#include <string>
struct CoolString {
    std::string s;
    CoolString(const char* p) : s(p){}
    char& operator[](size_t i) { return s.at(i); }
   operator char*() { return &s[0]; }
};

int main()  {
    CoolString ss("asd");
    ss[1] = '1';  // {1}
}
  

It looks valid, althought use of implicit user conversion operator to char* is a sign of bad design (std::string provides c_str instead). Above code compiles under g++ and clang, but fails to compile under Visual Studio. The error is that call {1} is ambiguous, and looks like below:

2>c:\prj\test.cpp(10) : error C2666: 'CoolString::operator []' : 2 overloads have similar conversions
2>        c:\prj\test.cpp(10): could be 'char &CoolString::operator [](size_t)'
2>        or 'built-in C++ operator[(char *, int)'
2>        while trying to match the argument list '(CoolString, int)'

so, compiler is unable to decide which one of those to choose:

ss.operator[](1) // line 1
ss.operator char*()[1] // line 2

In case of second choice ( line 2 ), compiler wants to apply built in subscript operator after user defined conversion to char*. Built in subscript operator takes index as a ptrdiff_t type. Because this is a typedef, lets check how it differs under g++/clang from VS:

std::cout << "size_t = " << typeid(size_t).name() << std::endl; // VS2013: unsigned int, g++: unsigned long
std::cout << "ptrdiff_t = " << typeid(ptrdiff_t).name() << std::endl; // VS2013: int, g++: long

Under VS ptrdiff_t is int, while under g++/clang its long. While size_t is always unsigned. Lets look what conversions are required by both candidates:

ss.operator[](1) – here only integral promotion of int to unsigned int or unsigned long is required

ss.operator char*()[1] – here on VS only user defined conversion must be applied to ss (so that operator char*() is called), since 1 is an int it matches ptrdiff_t which under VS is also int. But under g++/clang two conversions are requried: first user defined to char*, and then promotion of int to long.

Now its clear why VS shows error, implicit conversions sequences are ambiguous – both are of same lenghts, also they differ on different positions, but I am not clear about it here (exact standard wording) – you could argue that integral promotion should be more promoted than user defined conversion.

Under g++/clang built in operator is not choosen because it requires two implicit conversions, first is user defined conversion, second is promotion int to long. So compiler choses ss.operator[](1) which requires only integral promotion.

The solution is to:
1. Dont use implicit conversion operator to char*
2. Replace size_t with int in char& operator[](size_t i), this will make it a perfect match and top priority candidate.
3. Use ss[1u]

For more references:
http://stackoverflow.com/questions/1726740/c-error-operator-2-overloads-have-similar-conversions
– “C++ Templates – The Complete Guide” – B.2.1

Written by Marcin

January 17, 2015 at 3:10 am

Posted in C++, Uncategorized

String literal as non type template parameter

leave a comment »


Unfortunately it is not possible to have template with string literal as a non type parameter. Until C++11 reason was that string literals had internal linkage, while non type template parameters allowed only external linkage pointers. It makes sense because templ_class should have the same typeid in all translation units. It kind of could work with string pooling enabled – which is availabe for both gcc and msvc. After C++11 the reason is more of that string literals does not match any of the allowed entities as template parameter.

You can use character array initialized with string literal as non type parameter. It compiles both with g++ and msvc:

#include <typeinfo>

template<const char* STR>
class MyStr {};

extern const char my_string[]; const char my_string[] = "alfabet";
extern const char my_string2[]; const char my_string2[] = "alfabet";

// or use constexpr
//constexpr const char my_string[] = "alfabet";
//constexpr const char my_string2[] = "alfabet";

extern const char* my_string_pc;
const char* my_string_pc = "alfabet";

int main(){
   
    // This fails with:
    //  main.cpp:30:31: error: 'my_string_pc' is not a valid template argument because 'my_string_pc' is a variable, not the address of a variable
    //    typedef MyStr<my_string_pc> str_type3;
    typedef MyStr<my_string_pc> str_type3;     
   
    // Below compiles, but gives different types
    typedef MyStr<my_string> str_type1;   
    typedef MyStr<my_string2> str_type2; 
   
    // This fails
    static_assert(std::is_same<str_type1, str_type2>::value, "");
   
    return 0;
}

EDIT:
After writing it I have found in standard ([temp.arg.nontype]) the actual reasoning for all above:

[ Note: A string literal (2.14.5) does not satisfy the requirements of any of these categories and thus is not
an acceptable template-argument. [ Example:

template<class T, const char* p> class X {
/ ... /
};
X<int, "Studebaker"> x1; // error: string literal as template-argument
const char p[] = "Vivisectionist";
X<int,p> x2; // OK

—end example ] —end note ]

refs:
http://stackoverflow.com/questions/5547852/string-literals-not-allowed-as-non-type-template-parameters

Written by Marcin

December 30, 2014 at 12:56 am

Posted in C++, Uncategorized

Linkage of templates

leave a comment »


I would like to share observation of another non standard behavior found in Visual Studio compiler. According to C++ Templates Complete Guide: “class templates cannot share a name with a different kind of entity”, below code:

int X;
template
class X {    
};

gives following error with g++ 4.9:


main.cpp:8:7: error: 'template struct X' redeclared as different kind of symbol
class X {
^
main.cpp:6:5: note: previous declaration 'int X'
int X;

but with VS2013 compiles just fine.

Written by Marcin

December 28, 2014 at 2:24 am

Posted in Uncategorized

Be carefull with subnormal floats

leave a comment »


Subnormal floating point numbers are values that are very small (.0f to ~.1f), operations on them are specially treated by CPUs. Below is a timing of simple sumation of floating point values, you can see that once subnormal values are used, performance instantly drops 10 or more times.

http://coliru.stacked-crooked.com/a/6ffe763ab4b5ada0

// Example code for testing subnormal float efficency
float f = 2.0f;
std::chrono::steady_clock::duration prevdiff;
for (int k = 0; k < 10000; ++k) {
  float sum = .0f;		
  auto start = std::chrono::steady_clock::now();

  for (int n = 0; n < 1e7; ++n) {
    sum += f * 0.1f;  // interesting, but simple summation show no performance degradation
  }

  auto end = std::chrono::steady_clock::now();
  auto diff = end - start;
  std::cout << k << ", f=" << f << ", isnormal=" << std::isnormal(f)
    << ", elapsed=" << std::chrono::duration_cast<std::chrono::milliseconds>(diff).count() << "ms" <<
    ", ratio=" << (prevdiff.count() == 0 ? 0 : (diff.count() / prevdiff.count())) << std::endl;
    prevdiff = diff;

   if (f < std::numeric_limits<float>::min())
     break;

    f /= 2.0f;
}

here is the interesting part from the output:

123, f=1.88079e-37, isnormal=1, elapsed=69ms, ratio=1
124, f=9.40395e-38, isnormal=1, elapsed=799ms, ratio=11 <-------- here performance drop
125, f=4.70198e-38, isnormal=1, elapsed=800ms, ratio=1
126, f=2.35099e-38, isnormal=1, elapsed=800ms, ratio=0
127, f=1.17549e-38, isnormal=1, elapsed=796ms, ratio=0
128, f=5.87747e-39, isnormal=0, elapsed=985ms, ratio=1

whats interesting is that this drop happens before std::isnormal returns true. You can use _MM_SET_FLUSH_ZERO_MODE(_MM_FLUSH_ZERO_ON); to make CPU not do computations using subnormal values.

This post is based on following SO : http://stackoverflow.com/questions/9314534/why-does-changing-0-1f-to-0-slow-down-performance-by-10x.

Written by Marcin

December 21, 2014 at 4:40 pm

Posted in Uncategorized

RandomAccessFile does not lock your file by default

leave a comment »


If you use this class for writing then you probably want to ensure that you are the only one who is allowed to modify given file. I recently was fixing code that used RandomAccessFile for writing huge files downloaded from internet, but it didnt lock those file during writing. Whats wrong with that? Actually from the user perspective nothing, those file wont be accessed any way by any other software, but what we are missing here is the fact that only one thread should be allowed to read/write those files is an invariant which should be checked in code to trap any errors that could show up. And if this invariant were checked in the code I was fixing, developer would find his bug much earlier.

In todays heavily multithreaded environments it is easy to introduce errors, thats why invariants should be checked whether possible and exceptions should be thrown always when they are not correctly met.

Below is a small method for checking if file is locked or not.

public static boolean isFileLocked(File f) {
	RandomAccessFile file = null;
	FileChannel fileChannel = null;
	FileLock fileLock = null;

	try {
		file = new RandomAccessFile(f, "r");
		fileChannel = file.getChannel();
		if (fileChannel == null)
			return false;
		fileLock = fileChannel.lock();
	} catch (FileNotFoundException fnfe) {
		return false;
	} catch (Exception e) {
		// ClosedChannelException
		// NonWritableChannelException
		// OverlappingFileLockException
		// FileLockInterruptionException
		// AsynchronousCloseException
		// IOException
		return true;
	} finally {
		try {
			if (fileLock != null)
				fileLock.release();
			if (fileChannel != null)
				fileChannel.close();
			if (file != null)
				file.close();
		} catch (IOException ioe) {
			return false;
		}
	}
	return false;
}

Written by Marcin

October 4, 2012 at 3:31 pm

Posted in Android, Uncategorized

Moving to PreferenceActivity

leave a comment »


PreferenceActivity seems to be the best choice when it comes to allowing user edit application settings. If you started with custom dialog with lots of widgets for your preferences and now you want to move to PreferenceActivity then be prepared for some difficulties:

– your Shared Preferences keys must be of String type and values must be of String or Boolean types.
– if you have any int values in your preferences, then remember to convert it to string before PreferenceActivity will touch it, otherwise be prepared for exceptions
– you should not start PreferenceActivity derived Activity class using sub-activities, but rather use SharedPreferences.OnSharedPreferenceChangeListener and apply changes once preference changes, this might cause troubles if processing is time consuming
– if you used arrays.xml for your widget allowed values (ie. spinner), then make sure that allowed values are strings, ie.if you had 3 then change it to “3”. PreferenceActivity works with Strings – remember.
– and remember to read docs on PreferenceActivity, since its methods are heavily deprecated since HONEYCOMB release

Written by Marcin

December 26, 2011 at 1:59 am

Posted in Uncategorized

Tagged with