Savage Developers Guide
|
Savage Developers Guide |
|
The Savage Developers Guide describes system configuration supporting various software projects by the NPS MOVES Savage Research Group.
Ant | Chat | Cygwin | DES | DIS | Donate | Example X3D Models | Free | hamming | Java | JavaScript | Jenkins | Licensing | Media | MySQL | NetBeans | Projects | Security | Servers | Subversion | telnet | Time | Tomcat | Tools | Wikis | Writing | Xj3D | XML | XML Catalog | SavageDefense | X3D-Edit | X3D Resources | X3D Scene Authoring Hints | X3D Tooltips | Contact
SAVAGE stands for Scenario Authoring and Visualization for Advanced Graphics Environments. Our mascot is a LION because Like It Or Not, on the Web, open source and open standards always win!
Ant is a Java library and command-line tool used to automate common tasks for building projects. Ant task syntax is in XML, making task definition and validation much simpler than other build systems.
Ant is used to maintain most Savage projects. Key references include:
Recommended ant configuration details follow.
C:\apache-ant-1.10.15
PATH=.;%ANT_HOME%\bin;... (etc.)
ANT_HOME=C:\apache-ant-1.10.15
ANT_OPTS=-Xss16m -Xmx4096M
or
ANT_OPTS=-Xss16m -Xmx8192M
or
ANT_OPTS=-Xss16m -Xmx12228M
On Windows systems, set environment variables by selecting
Control Panel > System Properties > Advanced settings > Environment variables System variables
C:\apache-ant-1.10.15> ant -f fetch.xml -Ddest=system
ant -version Apache Ant(TM) version 1.10.15 compiled on August 25 2024
junit,
sshexec
and
scp
require the addition of external
library dependencies.
ANT_HOME,
then copy
ant-contrib-*.jar
into the
ANT_HOME/lib/
subdirectory.
Also be sure to check ownership and executable permissions for this file.
You can now use any of the
ant-contrib tasks
by including the following top-level statement in your build.xml file.
<taskdef resource="net/sf/antcontrib/antcontrib.properties"/>
xml-resolver.jar for resolving XML Catalog url alternatives on local file system.
xml-resolver.jar
in the
ANT_HOME/lib/
subdirectory.)
ANT_HOME/lib/
subdirectory.
{user.home}/.ant/lib
directory.
C:\apache-ant-1.10.15
scp task instead.
Further configuration options are described under Ant Setup.
(Note: this work is historical and not currently active.)
The Savage tools use chat software that supports the Extensible Messaging and Presence Protocol (XMPP), commonly known as Jabber chat.
xmpp://conference.savage.nps.edu
and the following rooms are publicly available:
Utility tools in the Cygwin distribution provide Unix command-line functionality for Windows systems. It is not needed for macOS or Linux systems.
C:\cygwin
ssh and sftp commands.
.;C:\cygwin64\bin;
near the beginning (before any Windows default directories)
CYGWIN=nodosfilewarning
/home/Username/ for files .bashrc .bash_profile and .inputrc)
The DES projects are a suite of integrated tools for analytic simulation. This work is entirely built using open-source Java, XML and X3D.
https://eos.nps.edu/svn/Simkit/trunk
with username guest
https://eos.nps.edu/wcm/DISKIT/trunk
with username guest
https://eos.nps.edu/wcm/Viskit/trunk
with username guest
The IEEE Distributed Interactive Simulation (DIS) protocol is an IEEE standard for conducting real-time platform-level simulation across multiple host computers. DIS is one of the most widely used modeling and simulation networking standards. DIS is used worldwide, especially by military organizations but also by other agencies such as those involved in space exploration and medicine.
Overview (from the standard). Data messages, known as Protocol Data Units (PDUs), that are exchanged on a network among simulation applications are defined. These PDUs are for interactions that take place within specified domains called protocol families, which include Entity Information/Interaction, Warfare, Logistics, Simulation Management, Distributed Emission Regeneration, Radio Communications, Entity Management, Minefield, Synthetic Environment, Simulation Management with Reliability, Information Operations, Live Entity Information/Interaction, and Non-Real-Time protocol.
Active Work. NPS is actively implementing complete coverage of the IEEE DIS version 7 protocol.
Key References
Savage software and models are openly published without charge and funded by research proposals. You are welcome to donate funds to support these efforts if you wish. You are welcome to tell our benefactors about your interest in Savage and X3D Graphics:
Please contact Don Brutzman (brutzman at nps.edu) if you are interested in sponsoring or partnering on further research.
The free cost of open-source software has been fundamentally important to the Savage group's ability to create applications and content that are highly capable, compliant with open standards, highly capable, maintainable and repeatable by others.
This long-term stability is essential to our accomplishing the NPS mission of graduate education and research. It further allows our graduates, and any other government agency, to utilize our standards-driven work compatibly with program life-cycle requirements and industry support.
The development and use of free software avoids common problems of "vendor lock in" through proprietary restrictions, inability of ongoing government efforts to fund the renewal of prior licenses, and inability to pay a company that is otherwise unwilling to perform software bugfixes and corrections.
This approach also avoids the common "short half-life" problems of companies that can either go out of business or else get acquired (and perhaps buried) by other companies. Such business pathologies not only break software but also negate correspondingly significant investments to create content of interest. In turn, such dependency on proprietary software leads to personnel vulnerabilities when an expert programmer or content developer can no longer apply their skills to improve prior accomplishments.
Further information on these software imperatives can be found in the Licensing section and through the following key sources.
hamming is the NPS supercomputer used for a variety of tasks with X3D Earth, Renderman rendering, and Viskit Design of Experiments.
This major asset is named for former NPS Emeritus Professor and distinguished computer scientist Richard W. Hamming.
Java is the primary programming language used in NPS Savage software projects.
To configure Java on your development system:
jdk-23
is default Java version for Savage projects, used
under a
non-viral open-source license.
docs directory tree provided by the .zip
into the directory containing the corresponding version of Java.
(Note that OpenJDK Javadoc
uses the same javadoc as Oracle Javadoc.)
jdk-19.0.1
and the (much earlier but commonplace)
jdk1.8.0_221.
These are tested under "Development Use" permitted by the
Oracle JDK license.
JAVA_HOME such as
JAVA_HOME=C:\Program Files\Java\openjdk\jdk-23 # OpenJDK, or
JAVA_HOME=C:\Program Files\Java\jdk-19.0.1 # Oracle JDK
PATH=.;%JAVA_HOME%\bin;%ANT_HOME%\bin;
PATH environment variable.
$ java -version # OpenJDK 23 openjdk version "23" 2024-09-17 OpenJDK Runtime Environment (build 23+37-2369) OpenJDK 64-Bit Server VM (build 23+37-2369, mixed mode, sharing)
$ java -version # Oracle JDK 19 java version "19.0.1" 2022-10-18 Java(TM) SE Runtime Environment (build 19.0.1+10-21) Java HotSpot(TM) 64-Bit Server VM (build 19.0.1+10-21, mixed mode, sharing)
Additional, older Java resources:
C:\Program Files\Java\jdk1.8.0_221\jre\lib\security
JavaScript is a common programming language used in Web browsers for HTML and X3D scripts. n.b. JavaScript is not the same as Java, they are different programming languages.
(Note: this work is historical and not currently active.)
The Savage Jenkins Build Servers are online continuous-integration servers that are used to test open-source code and content distributions. They automatically check out versioned projects and perform a variety of custom build tasks. This is an excellent capability that complements individual programmer efforts to support agile, effective team development.
Configuration setup for our jenkins servers includes installation of ant-contrib as described in the Ant section.
Helpful external references:
Wherever possible our group utilizes both open standards and open source licensing.
Motivation: as a government-funded university, it is important that our work be published publicly and made available freely. As researchers working with military students and worldwide communities of interest, we are striving to maximize the interoperability and 3D visualization capabilities of data streams used for situational awareness. Our strategy to accomplish these goals is the development, testing, adoption and evaluation of Web standards.
It is always important to observe Intellectual Property Rights (IPR) requirements. Because of our involvement in multiple groups developing open standards and open source, it is important that our work remains "untainted" and unexposed to software that may contain patent encumbrances. We do not want to jeopardize any of the standards or codebases that we work on.
In particular, we don't like to use tools with the following characteristics:
Rationale for success:
Open Technology Development (OTD) describes accepted best practices for open source and open standards, emphasizing business cases for government agencies, and providing excellent rationales and detailed guidance.
Contributors who wish to provide code or content for any of the archives we maintain can either use the Savage open-source license or else another open-source license of their choice. In this way, these public archives remain freely reusable.
3D graphics modeling and Web media benefit from using open-source content. The X3D Scene Authoring Hints contain detailed tips and guidelines for creating Web content. Here are some additional useful resources.
NetBeans has excellent support for MySQL databases. The NetBeans help page "Using MySQL with the IDE" is a quick reference. Other resources include:
Additional resources:
A secure shell (ssh) tunnel makes it possible to create a local connection to a remote database on another server.
This makes a port on your local host appear as if it is a port on the remote host. In reality, the traffic is first going into the port on the local host, then going across an encrypted link toRemoteHost.nps.edu, then being decrypted on the other side, and then sent to the MySQL port onRemoteHost. Finally the ssh tunnel returns the data response from the database to the port on your local host. This technique is secure because it lets us avoid having to unblock the firewall to open the MySQL port on the remote host. It also ensures that all the passwords and data sent across the network are encrypted. If you are connecting to an NPS database server, you will first need to be connected to the NPS intranet.
ssh -N username@RemoteHost.nps.edu -L 3307:localhost:3306sshUnix command for secure shell. sshsupport under Windows can be found as part of the Cygwin tools.-NDon't run any commands on the mmowgli host. Optional but nice. username@RemoteHost.nps.eduyour ssh login account on the remote host -LSet up a tunnel 3307:localhost:3306Set up a port on your system, 3307, that goes to the MySQL port, 3306, on the remote host.
We use 3307 in this case because there's often a development MySQL server running on
your system that's already using 3306.This technique is also helpful for locally listening to a Tomcat server port. For example:
ssh -N username -L 8080:tomcatServer.nps.edu:8181lets you monitor remote
tomcatServerresponse via your web browser athttp://localhost:8080Further
sshdetails are available using the Unix help manual page,man ssh.
Apache NetBeans is the Integrated Development Environment (IDE) used for most Savage project development.
Java Javadoc documentation is helpful, be sure to download a local copy of Java Javadoc as part of your current JDK release for faster performance.
docs directory tree provided by the .zip
into your local directory containing Java.
https://docs.oracle.com/en/java/javase/23/docs/api/index.html
there as well.
Once installed, default NetBeans configuration settings may need modification.
NETBEANS_HOME/etc/netbeans.conf
locations in the NetBeans installation directory tree:
C:\Program Files\NetBeans-23\netbeans\etc\netbeans.conf (default user settings)
netbeans.conf property
netbeans_jdkhome="C:\Program Files\Java\openjdk\jdk-23"
(for example).
netbeans_default_options environment variable:
-J--add-exports=jdk.internal.opt/jdk.internal.opt=ALL-UNNAMED
which, as reported by
NetBeans issue report,
avoids the compile-time error
Error starting modern compiler
netbeans_default_options environment variable:
-J-Djavax.xml.bind.context.factory=com.sun.xml.bind.v2.ContextFactory
netbeans_default_options environment variable:
-J--add-exports=java.xml/com.sun.org.apache.xerces.internal.impl.dv.util=ALL-UNNAMED
netbeans_default_options environment variable:
-J--illegal-access=warn
netbeans.conf file:
-J-Dsun.java2d.dpiaware=true
to
-J-Dsun.java2d.dpiaware=false
netbeans_default_options environment variable:
-J-Djdk.util.zip.disableZip64ExtraFieldValidation=true
Further configuration settings of interest:
x3d with Associatiated file type (MIME) selection XML Files (text/xml) to support X3D scene validation,
sch with Associatiated file type (MIME) selection XML Files (text/xml) to support Schematron rule-set validation.
The Savage group maintains a large number of open-source projects.
Extensible 3D (X3D) Graphics projects include the following:
The NPS MOVES team maintains multiple servers.
We typically use secure https://
rather than http://
for improved confidentiality between clients and servers.
Browser companies still! have not deployed DoD Certificate Authority (CA) links (apparently because they want the government to pay them to do so). Thus end users have to manually confirm each of the above certificates when first accessing these servers.
Nevertheless you can install DOD CA certificates locally (as shown in this Firefox dialog or, with some difficulty, Internet Explorer dialog).
If server keys are changed, you may need to clear previously cached values.
Cached certificate values can be found in directory ~/.ssh/known_hosts
or else invoke either
man ssh
or
man ssh-keygen for further details.
The NPS MOVES Savage team maintains multiple servers. Issue tracking is maintained at https://www.movesinstitute.org/bugzilla
Subversion is used for version control of Savage projects and files. Support for subversion is built into NetBeans and X3D-Edit.
NPS subversion servers:
Tutorial and resource information includes:
Some advanced subversion questions:
svn export [RepositoryURL]
relocate it using the https: protocol in order to meet security requirements for writing changes.
Use the subversion relocate command:
svn help relocate
The
telnet
protocol is widely used and provides a standard way of interfacing terminal-oriented devices and processes.
Given that security vulnerabilities may exist if used surreptitiously,
modern operating systems typically require specific configuration action to enable it.
ssh and telnet.
Also of interest:
ipconfig,
nc (netcat),
netstat,
tracert (traceroute),
whois.
(Of note is that netcat is also included in Cygwin, but might be quickly scrubbed by NPS antivirus routines for Windows.)
nc (netcat) which is functionally equivalent to ssh
and can be invoked from inside a terminal process.
Many ad hoc tools also exist (with lesser security) such as
brew.
Numerous simulations and network protocols depend on accurate time adjustment.
Apache Tomcat is an open-source implementation of the Java Servlet, JavaServer Pages (JSP), Java Expression Language and WebSocket technologies. Tomcat provides a "pure Java" HTTP web server environment in which Java code can run.
Developer installation notes:
CATALINA_HOME
points to root directory of current Tomcat installation, for example
C:\languages\java\apache-tomcat-10.1.20
nb-javac plugin is installed.
Example information for conf/tomcat-users.xml configuration file follows. Note that any username must also be configured in the Tomcat installation itself.
<role rolename="admin"/> <role rolename="admin-gui"/> <role rolename="admin-script"/> <role rolename="manager-gui"/> <role rolename="manager-script"/> <user username="moi" password="no-really" roles="admin,admin-gui,admin-script,manager-gui,manager-script" />
The Altova XML tools (XMLSpy, Diffdog, etc.) provide outstanding support for XML development.
NPS Downloadable Software and NPS MOVES IAGO software archive (NPS access only, password protected) provide a variety of locally licensed resources.
The following additional tools are also useful.
Wikis are an excellent way to collaboratively build websites online using a simple syntax. We typically use the same Wikimedia server software used by Wikipedia.
Writing up our accomplishments, in theses and reports, is crucially important. Here are some helpful resources.
Xj3D is an open-source browser and application programming interface written in Java for X3D and VRML97 graphics scenes.
Updated! Xj3D is upgraded to latest stable Java 23 (OpenJDK 23) using NetBeans 23
Source: https://gitlab.nps.edu/Savage/xj3d
Status: Regression testing complete, updated .jars (git) and installers are available, improvements continue.
Next steps: finish documentation updates and begin adding support for latest version X3D4. Have fun with Xj3D!
Continuing History
Applications using Xj3D include:
Recommended for developers. The most up-to-date Xj3D code repository is now available at:
https://gitlab.nps.edu/Savage/xj3d.git
Prior links for the Xj3D code repository supporting the long-running NPS branch:
Historic links for the original Xj3D trunk supports version 1.0 (released 15 April 2006).
http://svn.xj3d.org/xj3d_code
Also available:
The Extensible Markup Language (XML) is a simple text-based format for representing structured information: documents, data, configuration, books, transactions, invoices, and much more. It was derived from an older standard format called SGML (ISO 8879), in order to be more suitable for Web use.
An XML Catalog provides mappings from online URL or generic URI addresses to specific files on the local machine. This allows tools to refer to XML DTDs, schemas etc. to accomplish XML validation without being connected online.
OasisXmlCatalogX3D.xml provides schema and doctype address resolution for typical offline validation of X3D scenes.
Questions, suggestions and comments about these resources are welcome. Please send them to Don Brutzman (brutzman at nps.edu)
Available online at https://savage.nps.edu/developers.html
Revised 23 September 2024.
Official disclaimer:
Material contained herein is made available for the purpose of
peer review and discussion and does not necessarily reflect the
views of the Department of the Navy or the Department of Defense.