|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: INNER | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object | +--ec.Exchanger | +--ec.exchange.IslandExchange
IslandExchange is an Exchanger which implements a simple but quite functional asynchronous island model for doing massive parallel distribution of evolution across beowulf clusters. One of its really nice features is that because everything is in Java, your cluster can have mixed platforms in it (MacOS, Unix, Windoze, whatever you like). You can also have multiple processes running on the same machine, as long as they're given different client ports. IslandExchange operates over TCP/IP with Java sockets, and is compatible with checkpointing.
IslandExchange uses an arbitrary graph topology for migrating individuals from island (EC process) to island over the network. There are a few restrictions for simplicity, however:
Every island is a client. Additionally one island is designated a server. Note that, just like in the Hair Club for Men, the server is also a client. The purpose of the server is to synchronize the clients so that they all get set up properly and hook up to each other, then to send them small signal messages (like informing them that another client has discovered the ideal individual), and help them gracefully shut down. Other than these few signals which are routed through the server to the clients, all other information -- namely the migrants themselves -- are sent directly from client to client in a peer-to-peer fashion.
The topology of the network is stored solely in the server's parameter database. When the clients fire up, they first set up "Mailboxes" (where immigrants from other clients will appear), then they go to the server and ask it who they should connect to to send migrants. The server tells them, and then they then hook up. When a client has finished hooking up, it reports this to the server. After everyone has hooked up, the server tells the clients to begin evolution, and they're off and running.
Islands send emigrants to other islands by copying good individuals (selected with a SelectionMethod) and sending the good individuals to the mailboxes of receiving clients. Once an individual has been received, it is considered to be unevaluated by the receiving island, even though it had been previously evaluated by the sending island.
The IslandExchange model is typically asynchronous because migrants may appear in your mailbox at any time; islands do not wait for each other to complete the next generation. This is a more efficient usage of network bandwidth. When an island completes its breeding, it looks inside its mailbox for new migrants. It then replaces some of its newly-bred individuals (chosen entirely at random) with the migrants (we could have increased the population size so we didn't waste that breeding time, but we were lazy). It then flushes the mailbox, which patiently sits waiting for more individuals.
Clients may also be given different start times and modulos for migrating. For example, client A might be told that he begins sending emigrants only after generation 6, and then sends emigrants on every 4 generations beyond that. The purpose for the start times and modulos is so that not every client sends emigrants at the same time; this also makes better use of network bandwidth.
When a client goes down, the other clients deal with it gracefully; they simply stop trying to send to it. But if the server goes down, the clients do not continue operation; they will shut themselves down. This means that in general you can shut down an entire island model network just by killing the server process. However, if the server quits because it runs out of generations, it will wait for the clients to all quit before it finally stops.
IslandExchange works correctly with checkpointing. If you restart from a checkpoint, the IslandExchange will start up the clients and servers again and reconnect. Processes can start from different checkpoints, of course. However, realize that if you restart from a checkpoint, some migrants may have been lost in transit from island to island. That's the nature of networking without heavy-duty transaction management! This means that we cannot guarantee that restarting from checkpoint will yield the same results as the first run yielded.
Islands are not described in the topology parameters by their IP addresses; instead, they are described by "ids", strings which uniquely identify each island. For example, "gilligans-island" might be an id. :-) This allows you to move your topology to different IP addresses without having to change all your parameter files! You can even move your topology to totally different machines, and restart from previous checkpoints, and everything should still work correctly.
There are times, especially to experiment with dynamics, that you need a synchronous island model. If you specify synchronicity, the server's stated modulo and offset override any modulii or offsets specified by clients. Everyone will use the server's modulo and offset. This means that everyone will trade individuals on the same generation. Additionally, clients will wait until everyone else has traded, before they are permitted to continue evolving. This has the effect of locking all the clients together generation-wise; no clients can run faster than any other clients.
One last item: normally in this model, the server is also a client. But if for some reason you need the server to be a process all by itself, without creating a client as well, you can do that. You spawn such a server differently than the main execution of ECJ. To spawn a server on a given server params file (let's say it's server.params) but NOT spawn a client, you do:
java ec.exchange.IslandExchange -file server.params
...this sets up a special process which just spawns a server, and doesn't do all the setup of an evolutionary run. Of course as usual, for each of the clients, you'll run java ec.Evolve ... instead.
Parameters
Note: some of these parameters are only necessary for creating clients. Others are necessary for creating the server.
base.select classname, inherits and != ec.SelectionMethod |
client: The selection method used for picking migrants to emigrate to other islands |
base.select-to-die classname, inherits and != ec.SelectionMethod, default is ec.select.RandomSelection |
client: The selection method used for picking individuals to be replaced by incoming migrants. IMPORTANT Note. This selection method must not pick an individual based on fitness. The selection method will be called just after breeding but before evaluation; many individuals will not have had a fitness assigned at that point. You might want to design a SelectionMethod other than RandomSelection, however, to do things like not picking elites to die. |
base.server-addr String |
client: The IP address of the server |
base.server-port int >= 1 |
client: The port number of the server |
base.client-port int >= 1 |
client: The port number of the client (where it will receive migrants) -- this should be different from the server port. |
base.id String |
client: The "name" the client is giving itself. Each client should have a unique name. For example, "gilligans-island". |
base.i-am-server bool = true or false (default) |
client: Is this client also the server? If so, it'll read the server parameters and set up a server as well. |
base.sync bool = true or false (default) |
server: Are we doing a synchronous island model? If so, the server's modulo and offset override any client's stated modulo and offset. |
base.start bool = true or false (default) |
server: (Only if island model is synchronous) The generation when islands begin sending emigrants. |
base.mod bool = true or false (default) |
server: (Only if island model is synchronous) The number of generations islands wait between sending emigrants. |
base.num-islands int >= 1 |
server: The number of islands in the topology. |
base.island.n.id String |
server: The ID of island #n in the topology. |
base.island.n.num-mig int >= 1 |
server: The number of islands that island #n sends emigrants to. |
base.island.n.mig.m int >= 1 |
server: The ID of island #m that island #n sends emigrants to. |
base.island.n.size int >= 1 |
server: The number of emigrants (per subpopulation) that island #n sends to other islands. |
base.island.n.start int >= 0 |
server: The generation when island #n begins sending emigrants. |
base.island.n.mod int >= 1 |
server: The number of generations that island #n waits between sending emigrants. |
base.island.n.mailbox-capacity int >= 1 |
server: The maximum size (per subpopulation) of the mailbox for island #n. |
Parameter bases
base.select | selection method for the client's migrants |
Field Summary | |
Parameter |
base
My parameter base -- I need to keep this in order to help the server reinitialize contacts |
int |
clientPort
The port of the client mailbox |
static java.lang.String |
FOUND
Found signal |
static int |
FOUND_TIMEOUT
How long we sleep between checking for FOUND messages |
boolean |
iAmServer
whether the server should be running on the current island or not |
SelectionMethod |
immigrantsSelectionMethod
the selection method for emigrants |
SelectionMethod |
indsToDieSelectionMethod
the selection method for individuals to be replaced by immigrants |
int |
modulo
how often to send individuals |
int |
offset
after how many generations to start sending individuals |
static java.lang.String |
OKAY
Okay signal |
java.lang.String |
ownId
the id of the current island |
static java.lang.String |
P_CLIENT_PORT
The client port |
static java.lang.String |
P_IS_SERVER
Whether the server is also on this island |
static java.lang.String |
P_OWN_ID
The id of the island |
static java.lang.String |
P_SELECT_METHOD
The selection method for sending individuals to other islands |
static java.lang.String |
P_SELECT_TO_DIE_METHOD
The selection method for deciding individuals to be replaced by immigrants |
static java.lang.String |
P_SERVER_ADDRESS
The server address |
static java.lang.String |
P_SERVER_PORT
The server port |
java.lang.String |
serverAddress
The address of the server |
int |
serverPort
The port of the server |
java.lang.Thread |
serverThread
The thread of the server (is different than null only for the island with the server) |
int |
size
how many individuals to send each time |
static int |
SLEEP_TIME
How long we sleep in between attempts to connect or look for signals |
static java.lang.String |
SYNC
Synchronize signal |
boolean |
synchronous
synchronous or asynchronous communication |
Constructor Summary | |
IslandExchange()
|
Method Summary | |
void |
closeContacts(EvolutionState state,
int result)
Closes contacts with other processes, if that's what you're doing. |
void |
fireUpServer(EvolutionState state,
Parameter serverBase)
Fires up the server. |
void |
initializeContacts(EvolutionState state)
Initializes contacts with other processes, if that's what you're doing. |
static void |
main(java.lang.String[] args)
|
Population |
postBreedingExchangePopulation(EvolutionState state)
Performs exchanges after the population has been bred but before it has been evaluated, once every generation (or pseudogeneration). |
Population |
preBreedingExchangePopulation(EvolutionState state)
Performs exchanges after the population has been evaluated but before it has been bred, once every generation (or pseudogeneration). |
void |
reinitializeContacts(EvolutionState state)
Initializes contacts with other processes, if that's what you're doing. |
java.lang.String |
runComplete(EvolutionState state)
Called after preBreedingExchangePopulation(...) to evaluate whether or not the exchanger wishes the run to shut down (with ec.EvolutionState.R_FAILURE). |
void |
setup(EvolutionState state,
Parameter _base)
Sets up the object by reading it from the parameters stored in state, built off of the parameter base base. |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
public static final java.lang.String P_SERVER_ADDRESS
public static final java.lang.String P_SERVER_PORT
public static final java.lang.String P_CLIENT_PORT
public static final java.lang.String P_IS_SERVER
public static final java.lang.String P_OWN_ID
public static final java.lang.String P_SELECT_METHOD
public static final java.lang.String P_SELECT_TO_DIE_METHOD
public static final int SLEEP_TIME
public static final int FOUND_TIMEOUT
public static final java.lang.String OKAY
public static final java.lang.String SYNC
public static final java.lang.String FOUND
public java.lang.Thread serverThread
public Parameter base
public java.lang.String serverAddress
public int serverPort
public int clientPort
public boolean iAmServer
public java.lang.String ownId
public SelectionMethod immigrantsSelectionMethod
public SelectionMethod indsToDieSelectionMethod
public boolean synchronous
public int modulo
public int offset
public int size
Constructor Detail |
public IslandExchange()
Method Detail |
public static void main(java.lang.String[] args) throws java.lang.InterruptedException
public void setup(EvolutionState state, Parameter _base)
Setup
public void fireUpServer(EvolutionState state, Parameter serverBase)
public void initializeContacts(EvolutionState state)
initializeContacts
in class Exchanger
public void reinitializeContacts(EvolutionState state)
reinitializeContacts
in class Exchanger
public Population preBreedingExchangePopulation(EvolutionState state)
Exchanger
preBreedingExchangePopulation
in class Exchanger
public Population postBreedingExchangePopulation(EvolutionState state)
Exchanger
postBreedingExchangePopulation
in class Exchanger
public java.lang.String runComplete(EvolutionState state)
runComplete
in class Exchanger
public void closeContacts(EvolutionState state, int result)
closeContacts
in class Exchanger
|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: INNER | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |