mus_particle_comm_type_module Module

In mus_particle_comm_type_module the communication types and routines for particles are defined


Uses


Variables

Type Visibility Attributes Name Initial
integer, public :: mus_pIDvector_type
integer, public :: mus_wallPos_type
integer, public :: mus_particleInfo_type
integer, public :: mus_positionUpdate_type
integer, public :: mus_particleState_type

Derived Types

type, public, sequence  ::  mus_particles_pIDvector_type

process-wise buffer for particle force contribution data Used to exchange particle force or velocity data with a specific process

Components

Type Visibility Attributes Name Initial
real(kind=rk), public :: V(6)

Actual data vector. Used for: * force and moment contributions: V = (Fx, Fy, Fz, RFx, RFy, RFz) * velocity updates: V = (vx, vy, vz, RVx, RVy, RVz)
integer, public :: pID

Force contribution pertains to the particle with this ID

type, public ::  mus_particles_vectorbuffer_type

Components

Type Visibility Attributes Name Initial
type(mus_particles_pIDvector_type), public, allocatable :: val(:)

explicit buffer for data to be transferred
integer, public :: nParticles

number of particles to exchange, depends on number of particles at the interface of two processes. nParticles can change at every time step!

Read more…
integer, public :: maxParticles

type, public, sequence  ::  mus_particles_wallPos_type

process-wise buffer for wall position data Used to determine the average wall location using wall position sums over all processes

Components

Type Visibility Attributes Name Initial
real(kind=rk), public :: V(3)

Actual data vector. Used for: * wall position sums ( sum of x-coordinates, sum of y, sum of z )
integer, public :: I(2)

Wall position sum pertains to the particle with this ID * I(1) = nWallPos = number of elements in wallPosSum * I(2) = particleID to which this wallPosSum pertains

type, public ::  mus_particles_wallbuffer_type

Components

Type Visibility Attributes Name Initial
type(mus_particles_wallPos_type), public, allocatable :: val(:)

explicit buffer for data to be transferred
integer, public :: nParticles

number of particles to exchange, depends on number of particles at the interface of two processes. nParticles can change at every time step!

Read more…
integer, public :: maxParticles

type, public ::  mus_particles_IDbuffer_type

process-wise buffer for particle IDs Used to inform other processes when a particle needs to be removed/killed

Components

Type Visibility Attributes Name Initial
integer, public, allocatable :: val(:)

explicit buffer for data to be transferred
integer, public :: nParticles

number of particles to exchange, depends on number of particles at the interface of two processes. nParticles can change at every time step!

Read more…
integer, public :: maxParticles

type, public, sequence  ::  mus_particles_state_type

Components

Type Visibility Attributes Name Initial
real(kind=rk), public :: V(12)

Continuous particle data to be sent or received We gather all reals to be sent in one long vector. The order of the elements is: pos(1:6) = V(1:6) vel(1:6) = V(7:12)
integer, public :: I(5)

Also pack all integers to be sent or received in one vector coordOfOrigin = I(1:4) particleID = I(5)

type, public ::  mus_particles_statebuffer_type

Components

Type Visibility Attributes Name Initial
type(mus_particles_state_type), public, allocatable :: val(:)
integer, public :: nParticles

number of particles to exchange, depends on number of particles at the interface of two processes. nParticles can change at every time step!

Read more…
integer, public :: maxParticles

type, public, sequence  ::  mus_particles_positionUpdate_type

Components

Type Visibility Attributes Name Initial
real(kind=rk), public :: V(6)

Continuous particle data to be sent or received We gather all reals to be sent in one long vector. The order of the elements is: pos(1:6) = V(1:6)
integer, public :: I(5)

Also pack all integers to be sent or received in one vector coordOfOrigin = I(1:4) particleID = I(5)

type, public ::  mus_particles_posbuffer_type

Components

Type Visibility Attributes Name Initial
type(mus_particles_positionUpdate_type), public, allocatable :: val(:)
integer, public :: nParticles

number of particles to exchange, depends on number of particles at the interface of two processes. nParticles can change at every time step!

Read more…
integer, public :: maxParticles

type, public, sequence  ::  mus_particles_info_type

Components

Type Visibility Attributes Name Initial
real(kind=rk), public :: V(32)

Continuous particle data to be sent or received We gather all reals to be sent in one long vector. The order of the elements is: pos(1:6) = V(1:6) vel(1:6) = V(7:12) Fbuff(1,1:6) = V(13:18) Fbuff(2,1:6) = V(19:24) radius = V(25) mass = V(26) Fext(1:6) = V(27:32)
integer, public :: I(6)

ID of the particle that the data in V pertains to. Also pack all integers to be sent or received in one vector coordOfOrigin = I(1:4) Fnow = I(5) particleID = I(6)

type, public ::  mus_particles_infobuffer_type

Components

Type Visibility Attributes Name Initial
type(mus_particles_info_type), public, allocatable :: val(:)
integer, public :: nParticles

number of particles to exchange, depends on number of particles at the interface of two processes. nParticles can change at every time step!

Read more…
integer, public :: maxParticles

type, public ::  mus_particles_communication_type

Description of communication data for particles Will be a member of the particleGroup data type that exists on each process particleGroup%send for sendbuffer particleGroup%recv for recvbuffer

Components

Type Visibility Attributes Name Initial
integer, public :: nProcs = 0
integer, public, allocatable :: proc(:)

partition MPI rank
integer, public, allocatable :: rqHandle(:)

Request handle array
integer, public, allocatable :: rqHandle2(:)

2nd Request handle array so we can receive multiple KINDS of messages simultaneously
type(mus_particles_vectorbuffer_type), public, allocatable :: buf_force(:)

communication buffer for force contributions
type(mus_particles_posbuffer_type), public, allocatable :: buf_pos(:)

communication buffer for position updates (inc. coordOfOrigin)
type(mus_particles_vectorbuffer_type), public, allocatable :: buf_vec(:)

communication buffer for velocity updates
type(mus_particles_wallbuffer_type), public, allocatable :: buf_wall(:)

communication buffer for wall positions
type(mus_particles_statebuffer_type), public, allocatable :: buf_state(:)

communication buffer for particle state (= position + velocity + origin)
type(mus_particles_IDbuffer_type), public, allocatable :: buf_kill(:)

communication buffer for IDs of particles to be removed/killed
type(mus_particles_infobuffer_type), public, allocatable :: buf_particle(:)

communication buffer for all particle data together

Subroutines

public subroutine init_mus_particles_comm_type(me, nProcs, proc)

Allocate mus_particles_communication_type and its variables

Arguments

Type IntentOptional Attributes Name
type(mus_particles_communication_type), intent(inout) :: me

Communication type to initialize
integer, intent(in) :: nProcs

Number of neighboring processes with which to exchange particle data
integer, intent(in) :: proc(nProcs)

list of processes (ranks) to exchange with

public subroutine mus_particles_comm_init_vectorbuffer(me, maxParticles)

Arguments

Type IntentOptional Attributes Name
type(mus_particles_vectorbuffer_type), intent(inout) :: me
integer, intent(in) :: maxParticles

Number of particles to allocate space in buffer for This should be larger than the number of particles we expect to be at the interface of two processes at any time!

public subroutine mus_particles_comm_init_posbuffer(me, maxParticles)

Arguments

Type IntentOptional Attributes Name
type(mus_particles_posbuffer_type), intent(inout) :: me
integer, intent(in) :: maxParticles

Number of particles to allocate space in buffer for This should be larger than the number of particles we expect to be at the interface of two processes at any time!

public subroutine mus_particles_comm_init_wallbuffer(me, maxParticles)

Arguments

Type IntentOptional Attributes Name
type(mus_particles_wallbuffer_type), intent(inout) :: me
integer, intent(in) :: maxParticles

Number of particles to allocate space in buffer for This should be larger than the number of particles we expect to be at the interface of two processes at any time!

public subroutine mus_particles_comm_init_statebuffer(me, maxParticles)

Arguments

Type IntentOptional Attributes Name
type(mus_particles_statebuffer_type), intent(inout) :: me
integer, intent(in) :: maxParticles

Number of particles to allocate space in buffer for. This should be larger than the number of particles we expect to be at the interface of two processes at any time!

public subroutine mus_particles_comm_init_IDbuffer(me, maxParticles)

Arguments

Type IntentOptional Attributes Name
type(mus_particles_IDbuffer_type), intent(inout) :: me
integer, intent(in) :: maxParticles

Number of particles to allocate space in buffer for This should be larger than the number of particles we expect to be at the interface of two processes at any time!

public subroutine mus_particles_comm_init_particlebuffer(me, maxParticles)

Arguments

Type IntentOptional Attributes Name
type(mus_particles_infobuffer_type), intent(inout) :: me
integer, intent(in) :: maxParticles

Number of particles to allocate space in buffer for This should be larger than the number of particles we expect to be at the interface of two processes at any time!

public subroutine mus_particles_initForceContributionMPItype()

Arguments

None

public subroutine mus_particles_initWallPosMPItype()

Arguments

None

public subroutine mus_particles_initPositionUpdateMPItype()

Routine to create and commit the MPI type used to communicate position and coordOfOrigin

Arguments

None

public subroutine mus_particles_initParticleStateMPItype()

Routine to create and commit the MPI type used to communicate position, velocity and coordOfOrigin

Arguments

None

public subroutine mus_particles_initParticleInfoMPItype()

Routine to create and commit the MPI type used to communicate all continuous particle data

Arguments

None

public subroutine print_particles_comm(me)

Arguments

Type IntentOptional Attributes Name
type(mus_particles_communication_type), intent(inout) :: me

Communication type to print

public subroutine print_particles_pIDvectorbuffer(buff)

Arguments

Type IntentOptional Attributes Name
type(mus_particles_vectorbuffer_type), intent(in) :: buff

Communication type to print

public subroutine print_particles_statebuffer(buff)

Arguments

Type IntentOptional Attributes Name
type(mus_particles_statebuffer_type), intent(in) :: buff

Communication type to print

public subroutine find_particle_comm_procs4(prunedProcs, scheme, geometry, myRank, dpad)

find_particle_comm_procs generates a list of all processes we may need to communicate particle data with over the duration of the simulation. We only need to communicate this data for particles on our rank which also exist on other processes. This means we will never communicate with processes whose domains are not within one particle diameter of our domain. This is the key assumption used to generate the list of procs Hence the padding distance dpad should be set to the largest particle diameter in the simulation

Arguments

Type IntentOptional Attributes Name
type(dyn_intarray_type) :: prunedProcs

Dynamic array of procs which we will store the found procs in
type(mus_scheme_type), intent(in) :: scheme

Scheme for access to level descriptor
type(mus_geom_type), intent(in) :: geometry

Geometry information to determine TreeIDs of elements 'covered' by particle
integer, intent(in) :: myRank

This process's rank
real(kind=rk) :: dpad

Padding distance

public subroutine getSearchBoxLimits(elemCoord, elemPos, scheme, geometry, dirs, myRank, ndpad, lims)

getSearchBoxLimits is used in find_particle_comm_procs to determine where the non-local elements we have to check the owner proc of are.

Arguments

Type IntentOptional Attributes Name
integer, intent(in) :: elemCoord(4)

Coordinate to get search box around
integer, intent(in) :: elemPos

Element position in total list
type(mus_scheme_type), intent(in) :: scheme

Scheme for access to level descriptor
type(mus_geom_type), intent(in) :: geometry

Geometry information to determine whether halo has boundaries
integer, intent(in) :: dirs(6)

indices of cartesian directions [-x +x -y +y -z +z] in stencil
integer, intent(in) :: myRank

This procs rank
integer, intent(in) :: ndpad

Length of the search box in the directions where there are no local elements
integer, intent(out) :: lims(6)

lims = [xmin, xmax, ymin, ymax, zmin, zmax]

public subroutine pruneParticleCommProcs(procs, prunedProcs, nParts, myRank)

pruneParticleCommProcs takes the initial list of processes we think we need to communicate particle data with (determined using only local data) and checks whether those processes also think they need to communicate with us. The prunedProcs array contains the final list of processes for which we know we will send and also receive messages.

Arguments

Type IntentOptional Attributes Name
type(dyn_intarray_type) :: procs

Initial list of particle comm procs to be pruned
type(dyn_intarray_type) :: prunedProcs

Final (pruned) list of procs
integer, intent(in) :: nParts

number of processes
integer, intent(in) :: myRank

This procs rank