mozilla/security/coreconf/README

   1 OVERVIEW of "ns/coreconf":
   2
   3     This README file is an attempt to provide the reader with a simple
   4     synopsis of the "ns/coreconf" build system which was originally
   5     fundamentally designed and built to accomodate Netscape's binary
   6     release model.  Wherever possible, an attempt has been made to
   7     comply with the NSPR 2.0 build system, including mimicing the
   8     compiler/linker flags, and directory naming structure.  The reader
   9     should keep in mind that the system builds binary releases of
  10     header files, class files, libraries, and executables on numerous
  11     flavors of UNIX and Windows operating systems.  Unfortunately,
  12     no serious attempt has ever been made to incorporate an ability to
  13     generate cross-platform binaries on an Apple MacIntosh platform.
  14
  15     Note that this file will not attempt to redefine or document the
  16     architecture of this system.  However, documents on this subject
  17     are available at the following URL:
  18
  19         http://warp/hardcore/prj-ttools/specs/release/index.html
  20
  21
  22
  23 DEPENDENCIES of "ns/coreconf":
  24
  25     The "ns/coreconf" build system requires the specified versions of
  26     the following platform-dependent tools:
  27
  28         UNIX Platforms:
  29         --------------
  30         gmake (version 3.74 or later)
  31         perl 4.0 (NOTE:  perl 5.003 or later recommended)
  32         uname
  33
  34         Windows Platforms:
  35         -----------------
  36         gmake 3.74 (must use hacked Netscape version)
  37         shmsdos.exe (contained in Netscape gmake.exe)
  38         nsinstall.exe (contained in Netscape gmake.exe)
  39         perl.exe (version 4.0 for everything except testing;
  40                   NOTE:  MKS toolkit perl 5.002 is broken)
  41         perl5.exe (for testing;
  42                    NOTE:  perl 5.003 or later recommended;
  43                           MKS toolkit perl 5.002 is broken)
  44         uname.exe (use nstools version)
  45 \f
  46 ENHANCEMENTS to "ns/coreconf":
  47
  48     With the advent of Certificate Server 4.0 using the ns/coreconf
  49     build system, several changes had to be made to enhance
  50     ns/coreconf support for building Java/JNI classes/programs, as
  51     well as libraries slated to be released as binaries.  While the
  52     following may not represent an exhaustive list of these changes,
  53     it does attempt to be at least somewhat comprehensive:
  54
  55         (1) During the course of these enhancements, a total of
  56             four files have been modified, and four new files have
  57             been added.
  58
  59             The following files have been modified:
  60
  61                 - command.mk:    removed old definition of JAR
  62
  63                 - config.mk:     added include statement of new
  64                                  "jdk.mk" file
  65
  66                 - ruleset.mk:    allowed the $(MKPROG) variable to be
  67                                  overridden by supplying it with a
  68                                  default value of $(CC); augmented
  69                                  numerous definitions to enhance
  70                                  ability of ns/coreconf to produce
  71                                  a more robust set of libraries;
  72                                  added some JNI definitions; PACKAGE
  73                                  definition may be overridden by new
  74                                  "jdk.mk" file
  75
  76                 - rules.mk:      separated the compile phase of a
  77                                  program from the link phase of a
  78                                  program such that a developer can
  79                                  now strictly override program linkage
  80                                  by simply supplying a $(MKPROG)
  81                                  variable; augmented NETLIBDEPTH
  82                                  to use CORE_DEPTH but retain backward
  83                                  compatibility; added JNI section;
  84                                  modified .PRECIOUS rule;
  85
  86             The following files have been added:
  87
  88                 - README:        this file; an ASCII-based text
  89                                  document used to summarize the
  90                                  ns/coreconf build system and
  91                                  suitable (paginated) for printing
  92
  93                 - jdk.mk:        a file comprising most (if not all)
  94                                  of the default Java related build
  95                                  information; the definitions in this
  96                                  file are only included if NS_USE_JDK
  97                                  has been defined
  98
  99                 - jniregen.pl:   a perl script used to create a
 100                                  dependency for when JNI files should
 101                                  be regenerated (based upon any change
 102                                  to the ".class" file from which the
 103                                  ".h" file was originally generated)
 104
 105                 - outofdate.pl:  a perl script used to create a
 106                                  dependency for when ".class" files
 107                                  should be regenerated (based upon
 108                                  any change to the ".java" file
 109                                  from which the ".class" file was
 110                                  originally generated)
 111 \f
 112         (2) As stated above, the ns/coreconf build system now separates
 113             the link phase of a program from its compilation phase.
 114             While ns/coreconf still works exactly as it used to because
 115             the $(MKPROG) variable is assigned $(CC) by default, a developer
 116             may now override this behavior by simply supplying their
 117             own unique value for $(MKPROG) on every platform.  This allows
 118             a program compiled with $(CC) to link with external libraries
 119             that may contain "C++" linkage.  Before this change, a
 120             programmer would need to reference their own local copy of
 121             rules.mk (see the ns/sectools/cmd/pk12util program for
 122             an example of how this used to be accomplished).
 123
 124         (3) Currently, the ns/coreconf build system differs from the
 125             NSPR 2.0 build system which utilizes an "_s" to denote
 126             static libraries from import libraries.  In fact, the
 127             ns/coreconf build system adds no prefixes or suffixes to
 128             distinguish one version of static libraries from another.
 129             Note that both the ns/coreconf build system as well as the
 130             NSPR 2.0 build system do nothing to provide a method of
 131             distinguishing 16-bit from 32-bit static libraries on the
 132             same machine, either, since:
 133
 134                 a) this might only provide difficulty during
 135                    development, since static libraries always
 136                    need to be embedded within a program
 137                    (note this is highly unlikely, since libraries
 138                     for different platforms are subdivided via
 139                     a well-known subdirectory structure, and
 140                     a developer may use multiple trees for
 141                     development),
 142
 143                 b) this maintains backwards compatibility,
 144                    something very important since no legacy
 145                    programs will need to change their link phase, and
 146
 147                 c) Netscape as a company has dropped any plans
 148                    of future development of 16-bit products.
 149
 150         (4) Since several members of the Hardcore Security group did
 151             not favor NSPR 2.0's solution of adding an "_s" to static
 152             libraries on Windows platforms as a method to distinguish
 153             them from their import library cousins, a different solution
 154             was proposed and has been recently implemented for ns/coreconf:
 155
 156                 - a 16 has been added as a suffix to both dynamic and
 157                   import libraries built on 16-bit Windows platforms
 158
 159                 - a 32 has been added as a suffix to both dynamic and
 160                   import libraries built on 32-bit Windows platforms
 161
 162             Since the HCL release process currently only contains a
 163             single instance of building a dynamic library,
 164             ns/security/lib/fortcrypt/fort12.dll, the impact of this
 165             change should be relatively small.  (Note: HCL was the
 166             old name of NSS.)
 167
 168             It should be noted that although this would additionally
 169             limit the 8.3 namespace on 16-bit platforms, it is highly
 170             unlikely that any future development will be performed on
 171             this platform.
 172 \f
 173         (5) The $(LIBRARY_VERSION) tag has been added to all non-static
 174             libraries created on UNIX operating systems to alleviate
 175             any future confusion for binary releases which utilize this
 176             tag.  Again, it should be noted that this tag is only
 177             utilized on non-static libraries, since more than one
 178             version of the library may need to exist simultaneously
 179             if multiple products are utilized.
 180
 181             Currently, only one HCL released library utilizes this tag:
 182
 183                 ns/security/lib/fortcrypt/fort12.a
 184                 (e. g. - in this library, the tag has been set to '12')
 185
 186             Again, it should be noted that although this would
 187             additionally limit the 8.3 namespace on 16-bit platforms,
 188             it is highly unlikely that any future development will be
 189             performed on this platform.
 190
 191         (6) The $(JDK_DEBUG_SUFFIX) extension has been added to all
 192             library and program names to support debug versions of
 193             Java programs (e. g. - java_g, javac_g, etc).
 194
 195             Once again, it should be noted that although this would
 196             additionally limit the 8.3 namespace on 16-bit platforms,
 197             it is highly unlikely that any future Java development
 198             will be performed on this platform.
 199
 200         (7) Most (if not all) default definitions for java have been
 201             encapsulated within their own file, jdk.mk, which is
 202             always included by default in ns/coreconf/config.mk.
 203             However, the definitions within this file are only ever
 204             activated if NS_USE_JDK has been set to be 1.
 205
 206
 207         (8) Two perl scripts (jniregen.pl and outofdate.pl) have been
 208             added to the system to foster a more robust development
 209             environment for composing Java and JNI programs
 210             utilizing the ns/coreconf build system.  Both of these
 211             perl scripts are related to resolving dependencies which
 212             can not be accomplished through normal makefile dependencies.
 213
 214         (9) This file, README, was created in an attempt to allow
 215             developers who have familiarity with ns/coreconf a simple
 216             roadmap for what has changed, as well as a top-level view of
 217             what comprises ns/coreconf.  This file was written in
 218             ASCII (rather than HTML) primarily to promote simple
 219             paginated printing.
 220 \f
 221 OVERVIEW of "config.mk":
 222
 223     This file contains the configuration information necessary to
 224     build each "Core Components" source module:
 225
 226         include file name       Purpose
 227         ===================     =======================================
 228         arch.mk                 source and release <architecture> tags
 229
 230         command.mk              default command macros
 231                                 (NOTE: may be overridden in $(OS_CONFIG).mk)
 232
 233         $(OS_CONFIG).mk         <architecture>-specific macros
 234                                 (dependent upon <architecture> tags)
 235
 236         tree.mk                 release <tree> tags
 237                                 (dependent upon <architecture> tags)
 238
 239         module.mk               source and release <component> tags
 240                                 (NOTE:  A component is also called a module
 241                                 or a subsystem.  This file is dependent upon
 242                                 $(MODULE) being defined on the command
 243                                 line, as an environment variable, or in
 244                                 individual makefiles, or more
 245                                 appropriately, manifest.mn)
 246
 247         version.mk              release <version> tags
 248                                 (dependent upon $(MODULE) being defined on
 249                                 the command line, as an environment variable,
 250                                 or in individual makefiles, or more
 251                                 appropriately, manifest.mn)
 252
 253         location.mk             macros to figure out binary code location
 254                                 (dependent upon <platform> tags)
 255
 256         source.mk               <component>-specific source path
 257                                 (dependent upon <user_source_tree>,
 258                                 <source_component>, <version>, and
 259                                 <platform> tags)
 260
 261         headers.mk              include switch for support header files
 262                                 (dependent upon <tree>, <component>, <version>,
 263                                 and <platform> tags)
 264
 265         prefix.mk               compute program prefixes
 266
 267         suffix.mk               compute program suffixes
 268                                 (dependent upon <architecture> tags)
 269 \f
 270         jdk.mk                  define JDK
 271                                 (dependent upon <architecture>,
 272                                 <source>, and <suffix> tags)
 273
 274         ruleset.mk              Master "Core Components" rule set
 275                                 (should always be the last file
 276                                 included by config.mk)
 277
 278
 279
 280 OVERVIEW of "rules.mk":
 281
 282     The "rules.mk" file consists of four sections.  The first section
 283     contains the "master" build rules for all binary releases.  While
 284     this section can (and should) largely be thought of as "language"
 285     independent, it does utilize the "perl" scripting language to
 286     perform both the "import" and "release" of binary modules.
 287
 288     The rules which dwell in this section and their purpose:
 289
 290
 291         CATEGORY/rule::         Purpose
 292         ===================     =======================================
 293
 294         GENERAL
 295         -------
 296         all::                   "default" all-encompassing rule which
 297                                 performs "export libs program install"
 298
 299         export::                recursively copy specified
 300                                 cross-platform header files to the
 301                                 $(SOURCE_XPHEADERS_DIR) directory;
 302                                 recursively copy specified
 303                                 machine-dependent header files to the
 304                                 $(SOURCE_MDHEADERS_DIR) directory;
 305                                 although all rules can be written to
 306                                 repetively "chain" into other sections,
 307                                 this rule is the most commonly used
 308                                 rule to "chain" into other sections
 309                                 such as Java providing a simple
 310                                 mechanism which allows no need for
 311                                 developers to memorize specialized
 312                                 rules
 313
 314         libs::                  recursively build
 315                                 static (archival) $(LIBRARY), shared
 316                                 (dynamic link) $(SHARED_LIBRARY),
 317                                 and/or import $(IMPORT_LIBRARY)
 318                                 libraries
 319
 320         program::               recursively build $(PROGRAM)
 321                                 executable
 322
 323         install::               recursively copy all libraries to
 324                                 $(SOURCE_LIB_DIR) directory;
 325                                 recursively copy all executables to
 326                                 $(SOURCE_BIN_DIR) directory
 327
 328         clean::                 remove all files specified in the
 329                                 $(ALL_TRASH) variable
 330
 331         clobber::               synonym for "clean::" rule
 332 \f
 333         realclean::             remove all files specified by
 334                                 $(wildcard *.OBJ), dist, and in
 335                                 the $(ALL_TRASH) variable
 336
 337         clobber_all::           synonym for "realclean::" rule
 338
 339         private_export::        recursively copy specified
 340                                 cross-platform header files to the
 341                                 $(SOURCE_XPPRIVATE_DIR) directory
 342
 343
 344         IMPORT
 345         ------
 346         import::                uses perl script to retrieve specified
 347                                 VERSION of the binary release from
 348                                 $(RELEASE_TREE)
 349
 350         RELEASE
 351         -------
 352         release_clean::         remove all files from the
 353                                 $(SOURCE_RELEASE_PREFIX) directory
 354
 355         release::               place specified VERSION of the
 356                                 binary release in the appropriate
 357                                 $(RELEASE_TREE) directory
 358
 359         release_export::        recursively copy specified
 360                                 cross-platform header files to the
 361                                 $(SOURCE_XPHEADERS_DIR)/include
 362                                 directory
 363
 364         release_md::            recursively copy all libraries to
 365                                 $(SOURCE_RELEASE_PREFIX)/
 366                                 $(SOURCE_RELEASE_LIB_DIR) directory;
 367                                 recursively copy all executables to
 368                                 $(SOURCE_RELEASE_PREFIX)/
 369                                 $(SOURCE_RELEASE_BIN_DIR) directory
 370
 371         release_jars::          use perl script to package appropriate
 372                                 files in the $(XPCLASS_JAR),
 373                                 $(XPHEADER_JAR), $(MDHEADER_JAR), and
 374                                 $(MDBINARY_JAR) jar files
 375
 376         release_cpdistdir::     use perl script to copy the
 377                                 $(XPCLASS_JAR), $(XPHEADER_JAR),
 378                                 $(MDHEADER_JAR), and $(MDBINARY_JAR)
 379                                 jar files to the specified VERSION
 380                                 of the $(RELEASE_TREE) directory
 381
 382
 383
 384         TOOLS and AUTOMATION
 385         --------------------
 386         platform::              tool used to display the platform name
 387                                 as composed within the "arch.mk" file
 388
 389         autobuild::             automation rule used by "Bonsai" and
 390                                 "Tinderbox" to automatically generate
 391                                 binary releases on various platforms
 392
 393         tests::                 automation tool used to run the
 394                                 "regress" and "reporter" tools
 395                                 on various regression test suites
 396 \f
 397     The second section of "rules.mk" primarily contains several
 398     "language" dependent build rules for binary releases.  These are
 399     generally "computed" rules (created on the "fly"), and include
 400     rules used by "C", "C++", assembly, the preprocessor, perl, and
 401     the shell.
 402
 403     The rules which dwell in this section and their purpose:
 404
 405
 406         CATEGORY/rule::                   Purpose
 407         ===================               =============================
 408
 409         LIBRARIES
 410         ---------
 411         $(LIBRARY):                       build the static library
 412                                           specified by the $(LIBRARY)
 413                                           variable
 414
 415         $(IMPORT_LIBRARY):                build the import library
 416                                           specified by the
 417                                           $(IMPORT_LIBRARY) variable
 418
 419         $(SHARED_LIBRARY):                build the shared
 420                                           (dynamic link) library
 421                                           specified by the
 422                                           $(SHARED_LIBRARY) variable
 423
 424
 425         PROGRAMS
 426         --------
 427         $(PROGRAM):                       build the binary executable
 428                                           specified by the $(PROGRAM)
 429                                           rule
 430
 431         $(OBJDIR)/
 432         $(PROG_PREFIX)%.pure:             build the "purified" binary
 433                                           executable specified by this
 434                                           rule
 435
 436
 437         OBJECTS
 438         -------
 439         $(OBJDIR)/
 440         $(PROG_PREFIX)%$(OBJ_SUFFIX):     build the object file
 441                                           associated with the
 442                                           makefile rule dependency:
 443
 444                                               %.c   = C file
 445                                               %.cpp = C++ file
 446                                               %.cc  = C++ file
 447                                               %.s   = assembly file
 448                                               %.S   = assembly file
 449
 450         $(OBJDIR)/
 451         $(PROG_PREFIX)%:                  (NOTE: deprecated rule)
 452                                           build the object file
 453                                           associated with the
 454                                           makefile rule dependency:
 455
 456                                               %.cpp = C++ file
 457 \f
 458         MISCELLANEOUS
 459         -------------
 460         $(DIRS)::                         specifies a helper method
 461                                           used by $(LOOP_THROUGH_DIRS)
 462                                           to recursively change
 463                                           directories and invoke
 464                                           $(MAKE)
 465
 466         %.i:                              build the preprocessor file
 467                                           associated with the
 468                                           makefile rule dependency:
 469
 470                                               %.c   = C file
 471                                               %.cpp = C++ file
 472
 473         %:                                process the specified file
 474                                           using the method associated
 475                                           with the makefile rule
 476                                           dependency:
 477
 478                                               %.pl = perl script
 479                                               %.sh = shell script
 480
 481         alltags:                          tool used to recursively
 482                                           create a "ctags"-style
 483                                           file for reference
 484
 485     The third section of "rules.mk' primarily contains several JAVA
 486     "language" build rules for binary releases.  These are also
 487     generally "computed" rules (created on the "fly").
 488
 489     The rules which dwell in this section and their purpose:
 490
 491
 492         CATEGORY/rule::                   Purpose
 493         ===================               =============================
 494         $(JAVA_DESTPATH)::                create directory specified
 495                                           as the Java destination path
 496                                           for where classes are
 497                                           deposited
 498
 499         $(JAVA_DESTPATH)/$(PACKAGE)::     create directories specified
 500                                           within the $(PACKAGE)
 501                                           variable
 502
 503         $(JMCSRCDIR)::                    create directory specified
 504                                           as the JMC destination path
 505
 506         $(JRI_HEADER_CFILES):             used to generate/regenerate
 507                                           JRI header files for "C"
 508
 509         $(JRI_STUB_CFILES):               used to generate/regenerate
 510                                           JRI stub files for "C"
 511
 512         $(JNI_HEADERS):                   used to generate/regenerate
 513                                           JNI header files for "C"
 514 \f
 515     The fourth section of "rules.mk" primarily contains miscellaneous
 516     build rules for binary releases.  Many of these rules are here to
 517     create new subdirectories, manage dependencies, and/or override
 518     standard gmake "Makefile" rules.
 519
 520     The rules which dwell in this section and their purpose:
 521
 522
 523         CATEGORY/rule::                   Purpose
 524         ===================               =============================
 525
 526         $(PUBLIC_EXPORT_DIR)::            create directory used to
 527                                           house public "C" header files
 528
 529         $(PRIVATE_EXPORT_DIR)::           create directory used to
 530                                           house private "C" header
 531                                           files
 532
 533         $(SOURCE_XP_DIR)/
 534         release/include::                 create directory used to
 535                                           house "C" header files
 536                                           contained in a release
 537
 538         $(MKDEPENDENCIES)::               for UNIX systems, create
 539                                           a directory used to house
 540                                           dependencies and utilize
 541                                           the $(MKDEPEND) rule to
 542                                           create them
 543
 544         $(MKDEPEND)::                     cd to the dependency
 545                                           directory and create them
 546
 547         depend::                          if $(OBJS) exist, perform the
 548                                           $(MKDEPEND) rule followed by
 549                                           the $(MKDEPENDENCIES) rule
 550
 551         dependclean::                     remove all files contained
 552                                           in the dependency repository
 553
 554         .DEFAULT:                         standard gmake rule
 555
 556         .SUFFIXES:                        standard gmake rule
 557
 558         .PRECIOUS:                        standard gmake rule
 559
 560         .PHONY:                           standard gmake rule
 561