Docker: how to curl from one container into another

I have a docker compose file that is similar to the following one:

version: "3"
services:
    web:
        image: nginx
        ports:
            - "80:80"
        volumes:
            - ./volumes/www/app1.example.com/html:/var/www/app1.example.com/html:z
            - ./volumes/www/app2.example.com/html:/var/www/app2.example.com/html:z
            - ./volumes/www/app3.example.com/html:/var/www/app3.example.com/html:z
        restart: always
    cron:
      build:
        context: .
        dockerfile: cron-dockerfile
      links:
        - php
        - web
    php:
      # unimportant
    db:
      # unimportant

as you see I have a web container with nginx serving webpages from three virtual hosts. For each I have a domain name specific to application name. I also have a cron container which is supposed to do some maintance jobs from time to time. It does it by executing some php script on web server. Before migrating to docker I used basicly:

 curl http://app1.example.com/some_maintance.php 

But with docker I always get host unreachable error.

After some investigation I found that I am able to ping app1.example.com from cron container but curl or wget does not reach it, also web which is supposed to be a link to nginx container from cron container has different ip than actual app1.example.com (for which I have registrated domain with physical ip address). I have access to any other domain names – http://www.google.com etc. is accessible by curl.

So the first solution I tried was to update \etc\hosts on cron container, I added 1.2.3.4 app1.example.com entry, where 1.2.3.4 is IP address of web container. And it worked. The problem is that from what I know modifying \etc\hosts is not encouraged by docker community. Instead you should use extra_hosts in docker compose yaml file, but you can specify in extra_hosts only explicit IP addresses, you cannot use container name (like web in my configuration). From what I know, if you need such feature then it is a sign you should start using custom networks. I tried to use custom networks but unfortunately with no luck.

Finally what worked was to make curl use IP address of my web container (where I can put web), and then pass domain name as -H parameter. So the command looks as follows:

curl -H'Host: app1.example.com' web/some_maintance.php

Advertisement

Platform dependend overload resolution ambiguity

Here I will show case when types used internally by compiler might cause ambiguities in overload resolution. Below is example code:

#include <string>
struct CoolString {
    std::string s;
    CoolString(const char* p) : s(p){}
    char& operator[](size_t i) { return s.at(i); }
   operator char*() { return &s[0]; }
};

int main()  {
    CoolString ss("asd");
    ss[1] = '1';  // {1}
}
  

It looks valid, althought use of implicit user conversion operator to char* is a sign of bad design (std::string provides c_str instead). Above code compiles under g++ and clang, but fails to compile under Visual Studio. The error is that call {1} is ambiguous, and looks like below:

2>c:\prj\test.cpp(10) : error C2666: 'CoolString::operator []' : 2 overloads have similar conversions
2>        c:\prj\test.cpp(10): could be 'char &CoolString::operator [](size_t)'
2>        or 'built-in C++ operator[(char *, int)'
2>        while trying to match the argument list '(CoolString, int)'

so, compiler is unable to decide which one of those to choose:

ss.operator[](1) // line 1
ss.operator char*()[1] // line 2

In case of second choice ( line 2 ), compiler wants to apply built in subscript operator after user defined conversion to char*. Built in subscript operator takes index as a ptrdiff_t type. Because this is a typedef, lets check how it differs under g++/clang from VS:

std::cout << "size_t = " << typeid(size_t).name() << std::endl; // VS2013: unsigned int, g++: unsigned long
std::cout << "ptrdiff_t = " << typeid(ptrdiff_t).name() << std::endl; // VS2013: int, g++: long

Under VS ptrdiff_t is int, while under g++/clang its long. While size_t is always unsigned. Lets look what conversions are required by both candidates:

ss.operator[](1) – here only integral promotion of int to unsigned int or unsigned long is required

ss.operator char*()[1] – here on VS only user defined conversion must be applied to ss (so that operator char*() is called), since 1 is an int it matches ptrdiff_t which under VS is also int. But under g++/clang two conversions are requried: first user defined to char*, and then promotion of int to long.

Now its clear why VS shows error, implicit conversions sequences are ambiguous – both are of same lenghts, also they differ on different positions, but I am not clear about it here (exact standard wording) – you could argue that integral promotion should be more promoted than user defined conversion.

Under g++/clang built in operator is not choosen because it requires two implicit conversions, first is user defined conversion, second is promotion int to long. So compiler choses ss.operator[](1) which requires only integral promotion.

The solution is to:
1. Dont use implicit conversion operator to char*
2. Replace size_t with int in char& operator[](size_t i), this will make it a perfect match and top priority candidate.
3. Use ss[1u]

For more references:
– http://stackoverflow.com/questions/1726740/c-error-operator-2-overloads-have-similar-conversions
– “C++ Templates – The Complete Guide” – B.2.1

String literal as non type template parameter

Unfortunately it is not possible to have template with string literal as a non type parameter. Until C++11 reason was that string literals had internal linkage, while non type template parameters allowed only external linkage pointers. It makes sense because templ_class should have the same typeid in all translation units. It kind of could work with string pooling enabled – which is availabe for both gcc and msvc. After C++11 the reason is more of that string literals does not match any of the allowed entities as template parameter.

You can use character array initialized with string literal as non type parameter. It compiles both with g++ and msvc:

#include <typeinfo>

template<const char* STR>
class MyStr {};

extern const char my_string[]; const char my_string[] = "alfabet";
extern const char my_string2[]; const char my_string2[] = "alfabet";

// or use constexpr
//constexpr const char my_string[] = "alfabet";
//constexpr const char my_string2[] = "alfabet";

extern const char* my_string_pc;
const char* my_string_pc = "alfabet";

int main(){
   
    // This fails with:
    //  main.cpp:30:31: error: 'my_string_pc' is not a valid template argument because 'my_string_pc' is a variable, not the address of a variable
    //    typedef MyStr<my_string_pc> str_type3;
    typedef MyStr<my_string_pc> str_type3;     
   
    // Below compiles, but gives different types
    typedef MyStr<my_string> str_type1;   
    typedef MyStr<my_string2> str_type2; 
   
    // This fails
    static_assert(std::is_same<str_type1, str_type2>::value, "");
   
    return 0;
}

EDIT:
After writing it I have found in standard ([temp.arg.nontype]) the actual reasoning for all above:

[ Note: A string literal (2.14.5) does not satisfy the requirements of any of these categories and thus is not
an acceptable template-argument. [ Example:

template<class T, const char* p> class X {
/ ... /
};
X<int, "Studebaker"> x1; // error: string literal as template-argument
const char p[] = "Vivisectionist";
X<int,p> x2; // OK

—end example ] —end note ]

refs:
http://stackoverflow.com/questions/5547852/string-literals-not-allowed-as-non-type-template-parameters

Linkage of templates

I would like to share observation of another non standard behavior found in Visual Studio compiler. According to C++ Templates Complete Guide: “class templates cannot share a name with a different kind of entity”, below code:

int X;
template
class X {    
};

gives following error with g++ 4.9:


main.cpp:8:7: error: 'template struct X' redeclared as different kind of symbol
class X {
^
main.cpp:6:5: note: previous declaration 'int X'
int X;


but with VS2013 compiles just fine.

Be carefull with subnormal floats

Subnormal floating point numbers are values that are very small (.0f to ~.1f), operations on them are specially treated by CPUs. Below is a timing of simple sumation of floating point values, you can see that once subnormal values are used, performance instantly drops 10 or more times.

http://coliru.stacked-crooked.com/a/6ffe763ab4b5ada0

// Example code for testing subnormal float efficency
float f = 2.0f;
std::chrono::steady_clock::duration prevdiff;
for (int k = 0; k < 10000; ++k) {
  float sum = .0f;		
  auto start = std::chrono::steady_clock::now();

  for (int n = 0; n < 1e7; ++n) {
    sum += f * 0.1f;  // interesting, but simple summation show no performance degradation
  }

  auto end = std::chrono::steady_clock::now();
  auto diff = end - start;
  std::cout << k << ", f=" << f << ", isnormal=" << std::isnormal(f)
    << ", elapsed=" << std::chrono::duration_cast<std::chrono::milliseconds>(diff).count() << "ms" <<
    ", ratio=" << (prevdiff.count() == 0 ? 0 : (diff.count() / prevdiff.count())) << std::endl;
    prevdiff = diff;

   if (f < std::numeric_limits<float>::min())
     break;

    f /= 2.0f;
}

here is the interesting part from the output:

123, f=1.88079e-37, isnormal=1, elapsed=69ms, ratio=1
124, f=9.40395e-38, isnormal=1, elapsed=799ms, ratio=11 <-------- here performance drop
125, f=4.70198e-38, isnormal=1, elapsed=800ms, ratio=1
126, f=2.35099e-38, isnormal=1, elapsed=800ms, ratio=0
127, f=1.17549e-38, isnormal=1, elapsed=796ms, ratio=0
128, f=5.87747e-39, isnormal=0, elapsed=985ms, ratio=1

whats interesting is that this drop happens before std::isnormal returns true. You can use _MM_SET_FLUSH_ZERO_MODE(_MM_FLUSH_ZERO_ON); to make CPU not do computations using subnormal values.

This post is based on following SO : http://stackoverflow.com/questions/9314534/why-does-changing-0-1f-to-0-slow-down-performance-by-10x.

RandomAccessFile does not lock your file by default

If you use this class for writing then you probably want to ensure that you are the only one who is allowed to modify given file. I recently was fixing code that used RandomAccessFile for writing huge files downloaded from internet, but it didnt lock those file during writing. Whats wrong with that? Actually from the user perspective nothing, those file wont be accessed any way by any other software, but what we are missing here is the fact that only one thread should be allowed to read/write those files is an invariant which should be checked in code to trap any errors that could show up. And if this invariant were checked in the code I was fixing, developer would find his bug much earlier.

In todays heavily multithreaded environments it is easy to introduce errors, thats why invariants should be checked whether possible and exceptions should be thrown always when they are not correctly met.

Below is a small method for checking if file is locked or not.

public static boolean isFileLocked(File f) {
	RandomAccessFile file = null;
	FileChannel fileChannel = null;
	FileLock fileLock = null;

	try {
		file = new RandomAccessFile(f, "r");
		fileChannel = file.getChannel();
		if (fileChannel == null)
			return false;
		fileLock = fileChannel.lock();
	} catch (FileNotFoundException fnfe) {
		return false;
	} catch (Exception e) {
		// ClosedChannelException
		// NonWritableChannelException
		// OverlappingFileLockException
		// FileLockInterruptionException
		// AsynchronousCloseException
		// IOException
		return true;
	} finally {
		try {
			if (fileLock != null)
				fileLock.release();
			if (fileChannel != null)
				fileChannel.close();
			if (file != null)
				file.close();
		} catch (IOException ioe) {
			return false;
		}
	}
	return false;
}

Moving to PreferenceActivity

PreferenceActivity seems to be the best choice when it comes to allowing user edit application settings. If you started with custom dialog with lots of widgets for your preferences and now you want to move to PreferenceActivity then be prepared for some difficulties:

– your Shared Preferences keys must be of String type and values must be of String or Boolean types.
– if you have any int values in your preferences, then remember to convert it to string before PreferenceActivity will touch it, otherwise be prepared for exceptions
– you should not start PreferenceActivity derived Activity class using sub-activities, but rather use SharedPreferences.OnSharedPreferenceChangeListener and apply changes once preference changes, this might cause troubles if processing is time consuming
– if you used arrays.xml for your widget allowed values (ie. spinner), then make sure that allowed values are strings, ie.if you had 3 then change it to “3”. PreferenceActivity works with Strings – remember.
– and remember to read docs on PreferenceActivity, since its methods are heavily deprecated since HONEYCOMB release

putty and Ctrl + s that blocks screen

When you press Ctrl + s under putty, then your linux session will freeze and screen will not update. In my case I mostly had to restart my session which was quite annoying even though I use screen. Quicker solution is to press Ctrl + q. This is what many websites/blogs will tell you.

I am writing it here just to remind me of it, since I constantly forget that trick.

Hello world!

Hello everyone, just moved my WordPress blog from other server, here. I hope to stay here for a longer time.

printf("Hello world!");

I suppose I should say simply Welcome, but such title as above should give quick information what this blog is mainly about. I’m still in a process of moving from http://geekswithblogs.net/luskan, so this site will constantly undergo various changes. Quite recently I have bought cheap php + mysql enabled hosting, and now I feel like Christopher Columbus, having discovered a new land. It comes with Fantastico, which allows me among other things, to install in a few clicks : project management system – dotProject, FAQ list (faq.marcinj.prohost.pl) and also WordPress blog. I wish I had also gcc enabled ssh access, but hey its a cheap hosting!